Will Villanueva


Full Stack Software Engineer in the Bay Area, passionate about collaborating with other web enthusiasts.


The node.js Profiling Guide that Hasn’t Existed - Profiling node.js Applications - Part 1

Your application has grown significantly in traffic. Perhaps, you’re spawning multiple child processes and are working hard to conserve the memory consumption or CPU performance of your servers.

Maybe, your response-time has slowly decreased over time, and you have to keep resetting your servers.

Perhaps, you want to look cool and tell your friends about the latest and greatest in node profiling.

Well, I’m here to tell you that there are a lot of reasons to test your node applications, and if you aren’t profiling your applications, then you’re probably doing something wrong. It helps you write efficient code, and it keeps your eye on performance – Memory and CPU. It also makes you look cool. Don’t believe me? Go talk to your coworkers and you’ll see ☺

What to expect from this guide:

Recently, I was tasked with profiling an application we rebuilt using node.js. The application will see approximately 500,000 pageviews daily (from users), so performance is key. I was tasked with doing a bit of research on the available tools, and I learned a lot in the process.

I’m definitely no expert, but hopefully this article will give you a place to start, and point you in the right direction. A lot of the good resources are scattered on the web, and a lot of information is now out of date. I’ve vetted the resources out there, and I’ll give you my recommendation on the best tools along with some simple tips and tricks.

I will be releasing a series of posts on this topic. Here are the current releases:
Part2 – Finding a potential memory leak using memwatch

Part 3 - Finding The Cause of a Memory Leak Using Heap Snapshots

Part 4 – CPU and Performance Profiling - Coming soon...

Questions?

Please add comments below or reach out to me directly. My information is on my homepage, and I’d love to hear what you’re trying to accomplish. If I’m not sure how to help you, I know a lot of smart people who have gone deeper with node than most. I WILL bug them for you, that's a promise ☺

Starting Off:

First, I’ll list the helpful tools that are out there. There is a similar post on stack overflow that points you in the same direction. I’ll go deeper and point you to some other resources with updated information. I’ve listed the tools in the order of most helpful.

Most Useful:

Node-inspector - Node-inspector now has a heap and CPU profiler with the latest version (0.8.0).
The CPU profiler includes Flame Graphs, which is very exciting! It provides heap diffs, snapshots, and CPU profiling recording.

Node-Webkit-Agent - heap and CPU profiler using chrome tools. It provides heap diffs, snapshots, and CPU profiling recording

Memwatch - GC activity, heap usage, memory leak detection (GC-stats, node-gc are alternatives that are also helpful)

Flame Graphs - profile CPU usage – now compatible with linux. It previously only worked with illumos based systems. For Mac compatibility, use node-inspector's built in flame graphs

Apache Bench - You’ll need this to simulate a load and gather statistics from multiple queries

Look - CPU, heap, and performance/http profiling

Nodetime - Look is based off nodetime. This is a paid solution that requires you to share your data with their platform, but is a very useful tool

Somewhat Useful:

Webstorm 9 EAP Spy-JS - It has a lot of issues with larger applications. I’ll go into this later, and would not recommend it as a primary tool yet. It has a lot of promise, but mostly just offers tracing.

Not Too Helpful:

A lot of resources out there are misinformed. For example, until recently, flame graphs were not supported on anything other than an illumos-based system.

Trace GL – deprecated and no longer supported with newer version of node

Node-profiler – DEPRECATED!

Undecided:

StrongLoop - I’ve heard a lot of mixed reviews on the helpfulness of StrongLoop’s tools. Until they give me full access to their tools so I can review it, I can only go off of popular opinion. I’ve heard their profiling and performance tools could use a bit of improvement, but would love to be proved wrong ☺

The Future:

Webstorm is improving on a regular-basis, and I have hope that their tools will eventually kick a bit of ass.

Google's tracing framework – I expect amazing solutions will show up using this framework in the future. The website recommends node.js usage, and it just needs a bit of developer activity

Part 1 - Profiling node.js Applications

Part 2 - Finding a potential memory leak using memwatch

Part 3 - Finding The Cause of a Memory Leak Using Heap Snapshots

Part 4 – CPU and Performance Profiling - Coming soon...

comments powered by Disqus
comments powered by Disqus