Google I/O 2013 Session
We presented this material at Google I/O 2013. Check out the video below:
Gmail, we have a problem...
Memory Management Basics
- Number (e.g. 4, 3.14159)
- Boolean (true or false)
- String (“Hello World”)
These primitive types cannot reference any other values. In the object graph these values are always leaf or terminating nodes, meaning they never have an outgoing edge.
What About Arrays?
- Value - An instance of a primitive type, Object, Array, etc.
- Variable - A name that references a value.
- Property - A name in an Object that references a value.
When Does a Value Become Garbage?
A value becomes garbage when there is no path from a root to the value. In other words, starting at the roots and exhaustively searching all Object properties and variables that are alive in the stack frame, a value cannot be reached, it has become garbage.
email.message = document.createElement(“div”); displayList.appendChild(email.message);
And later, you remove the element from the display list:
As long as
What is Bloat?
Your page is bloated when you’re using more memory than necessary for optimal page speed. Indirectly, memory leaks also cause bloat but that is not by design. An application cache that does not have any size bound is a common source of memory bloat. Also, your page can be bloated by host data, for example, pixel data loaded from images.
What is Garbage Collection?
V8 Garbage Collector in Detail
To help further understand how garbage collection happens, let’s take a look at the V8 garbage collector in detail. V8 uses a generational collector. Memory is divided into two generations: the young and the old. Allocation and collection within the young generation is fast and frequent. Allocation and collection within the old generation is slower and less frequent.
V8 uses a two generation collector. The age of an value is defined as the number of bytes allocated since it was allocated. In practice, the age of an value is often approximated by the number of young generation collections that it survived. After a value is sufficiently old it is tenured into the old generation.
In practice, freshly allocated values do not live long. A study of Smalltalk programs, showed that only 7% of values survive after a young generation collection. Similar studies across runtimes found that on average between, 90% and 70% of freshly allocated values are never tenured into the old generation.
The young generation heap in V8 is split into two spaces, named from and to. Memory is allocated from the to space. Allocating is very fast, until, the to space is full at which point a young generation collection is triggered. Young generation collection first swaps the from and to space, the old to space (now the from space) is scanned and all live values are copied into the to space or tenured into the old generation. A typical young generation collection will take on the order of 10 milliseconds (ms).
Intuitively, you should understand that each allocation your application makes brings you closer exhausting the to space and incurring a GC pause. Game developers, take note: to ensure a 16ms frame time (required to achieve 60 frames per second), your application must make zero allocations, because a single young generation collection will eat up most of the frame time.
The old generation heap in V8 uses a mark-compact algorithm for collection. Old generation allocations occur whenever a value is tenured from the young generation to the old generation. Whenever an old generation collection occurs a young generation collection is done as well. Your application will paused on the order of seconds. In practice this is acceptable because old generation collections are infrequent.
V8 GC Summary
Over the past year, numerous features and bug fixes have made their way into the Chrome DevTools making them more powerful than ever. In addition, the browser itself made a key change to the performance.memory API making it possible for Gmail and any other application to collect memory statistics from the field. Armed with these awesome tools, what once seemed like an impossible task soon became an exciting game of tracking down culprits.
Tools and Techniques
Field Data and performance.memory API
As of Chrome 22, the performance.memory API is enabled by default. For long-running applications like Gmail, data from real users is invaluable. This information allows us to distinguish between power users-- those who spend 8-16 hours a day on Gmail, receiving hundreds of messages a day-- from more average users who spend a few minutes a day in Gmail, receiving a dozen or so messages a week.
This API returns three pieces of data:
- usedJSHeapSize - The amount of memory (in bytes) currently being used.
One thing to keep in mind is that the API returns memory values for the entire Chrome process. Although it is not the default mode, under certain circumstances, Chrome may open multiple tabs in the same renderer process. This means that the values returned by performance.memory may contain the memory footprint of other browser tabs in addition to the one containing your app.
Measuring Memory At Scale
Beyond tracking purposes, the field measurements also provide a keen insight into the correlation between memory footprint and application performance. Contrary to the popular belief that “more memory results in better performance”, the Gmail team found that the larger the memory footprint, the longer latencies were for common Gmail actions. Armed with this revelation, they were more motivated than ever to rein in their memory consumption.
Identifying a Memory Problem with the DevTools Timeline
The first step in solving any performance problem is to prove that the problem exists, create a reproducible test, and take a baseline measurement of the problem. Without a reproducible program, you cannot reliably measure the problem. Without a baseline measurement you don’t know by how much you’ve improved performance.
Proving a problem exists
Start by identifying a sequence of actions you suspect to be leaking memory. Start recording the timeline, and perform the sequence of actions. Use the trash can button at the bottom to force a full garbage collection. If, after a few iterations, you see a sawtooth shaped graph, you are allocating lots of shortly lived objects. But if the sequence of actions is not expected to result in any retained memory, and the DOM node count does not drop down back to the baseline where you began, you have good reason to suspect there is a leak.
Once you’ve confirmed that the problem exists, you can get help identifying the source of the problem from the DevTools Heap Profiler.
Finding Memory Leaks with the DevTools Heap Profiler
The Profiler panel provides both a CPU profiler and a Heap profiler. Heap profiling works by taking a snapshot of the object graph. Before a snapshot is taken both the young and old generations are garbage collected. In other words, you will only see values which were alive when the snapshot was taken.
There is too much functionality in the the Heap profiler to cover sufficiently in this article, but detailed documentation can be found on the Google Developers site. We’ll focus here on the all new Object Tracker, available now via an experiment on the latest Chrome builds. To enable the Object Tracker:
- Make sure you have the latest Chrome Canary.
- Navigate to about:flags, and make sure “Enable Developer Tools experiments” is enabled. You will need to restart Chrome to activate the experiments.
- Open the Developer Tools and click on the gear icon in the lower right. Find the Experiments item in the menu on the left, and check the box labeled “Enable heap objects tracking profile type”. You will need to close and reopen the DevTools to make this change take effect.
- Now, when you open the Profiler panel, you should see a fourth profile type option, called “Track Allocations”.
Using the Object Tracker
The Object Tracker combines the detailed snapshot information of the Heap Profiler with the incremental updating and tracking of the Timeline panel. Start a recording, perform a sequence of actions, then stop the recording for analysis. The Object Tracker takes heap snapshots periodically throughout the recording (as frequently as every 50 ms!) and one final snapshot at the end of the recording.
The bars at the top indicate when new objects are found in the heap. The height of each bar corresponds to the size of the recently allocated objects, and the color of the bars indicate whether or not those objects are still live in the final heap snapshot: blue bars indicate objects that are still live at the end of the timeline, gray bars indicate objects that were allocated during the timeline, but have since been garbage collected.
In the example above, an action was performed 10 times. The sample program caches five objects, so the last five blue bars are expected. But the leftmost blue bar indicates a potential problem. You can then use the sliders in the timeline above to zoom in on that particular snapshot and see the objects that were recently allocated at that point. Clicking on a specific object in the heap will show its retaining tree in the bottom portion of the heap snapshot. Examining the retaining path to the object should give you enough information to understand why the object was not collected, and you can make the necessary code changes to remove the unnecessary reference.
Resolving Gmail's Memory Crisis
By using the tools and techniques discussed above, the Gmail team was able to identify a few categories of bugs: unbounded caches, infinitely growing arrays of callbacks waiting for something to happen that never actually happens, and event listeners unintentionally retaining their targets. By fixing these issues, the overall memory usage of Gmail was dramatically reduced. Users in the 99% percent used 80% less memory than before and the memory consumption of the median users dropped by nearly 50%.
Because Gmail used less memory the GC pause latency was reduced, increasing the overall user experience.
Also of note, with the Gmail team collecting statistics on memory usage, they were able to uncover garbage collection regressions inside Chrome. Specifically, two fragmentation bugs were discovered when Gmail’s memory data began showing a dramatic increase in the gap between total memory allocated and live memory.
Call to Action
Ask yourself these questions:
- How much memory is my app using?
It’s possible that you are using too much memory which contrary to popular belief has a net negative on overall application performance. It’s hard to know exactly what the right number is, but, be sure to verify that any extra caching your page is using has a measurable performance impact.
- Is my page leak free?
If your page has memory leaks it can not only impact your page's performance but other tabs as well. Use the object tracker to help narrow in on any leaks.
- How frequently is my page GCing?
You can see any GC pause using Timeline panel in Chrome Developer Tools. If your page is GCing frequently, chances are you are allocating too frequently, churning through your young generation memory.