Using Xcode OpenGL ES Frame Capture

迈不过友情╰ 2022-08-20 10:13 15阅读 0赞

## [Using Xcode OpenGL ES Frame Capture][] ##

November 20, 2012

[tweet][]

![Xcode OpenGL ES Frame Capture][]

Having used Microsoft Visual Studio a while ago, I think that the real difference with Xcode is that Xcode comes with a full suite of performance tools *for free*. [Instruments][] is the most obvious one, but since version 4.5, Xcode includes a facility for capturing and analysing OpenGL ES frames. This tool, called OpenGL ES Frame Capture, is really simple and powerful: you can log all the OpenGL calls for a frame, inspects the content of OpenGL buffers, sees your OpenGL ES 2.0 shaders (even the “hidden” ones used by `GLKBaseEffect`). This is a quick review about how to use it in your app.

To follow this review and test a sample code, you can use one of the excellent [Ray Wenderlich][] OpenGL/GLKit tutorials like [OpenGL ES 2.0 for iPhone Tutorial][] or [How To Create A Simple 2D iPhone Game with OpenGL ES 2.0 and GLKit][].

*Disclaimer*: I’m a completely OpenGL rookie so I might employ not the right terminology in the following post. That’s said, let’s go!

### 1. How to setup OpenGL ES Frame Capture ###

There is a short [Apple documentation about OpenGL ES Frame Capture][]. To capture OpenGL frames, you’ll need to use, at least, Xcode 4.5. **The capture can only be done on a real device, running at minimum iOS 5**, and connected to your Mac.

If you’re already including GLKit or OpenGL framework in your code, your project is already set up for OpenGL frames capturing. Anyway, you can check it by going to Product > Manage Schemes…, double-click you app scheme, click on Edit….

![Enable capture][]

Tap on the Run scheme, then Options tab and check that ’OpenGL ES Frame Capture’ is set to Automatically Enabled or Enabled.

### 2. Capturing a Frame ###

You can capture a frame while you’re debugging your app. Launch the app in Debug mode (Product > Run or ⌘R). Once the app is launched, display the Debug Navigator (View > Navigators > Show Debug Navigator), and click on the FPS section of the Debug Navigator.

[![Show Debug Navigator][]][Show Debug Navigator 1]

You can watch the FPS updated while playing with your app and test different use cases. Your mission, if you accept it, is to maintain a good 60 FPS. On my app sample, I’m currently running at 36 FPS so I will need to find where the performance bottleneck is.

![Debug Navigator zoomed][]36 fps is not good, I would like to have 60 fps!

In the editor, you can see the time spent in the CPU and the GPU. If you want to achieve 60 FPS, you basically have 16 ms to update and render each frame. In my app, I’m spending 27 ms in the CPU and only 5 ms in the GPU.

![CPU vs GPU][]

Given this number, I can already see that my problems are not on the GPU side, but maybe simply in my code. Let’s capture a trace to have a better view of what’s happening at each frame rendering. Just above the Debug console, tap on the Camera icon to capture an entire frame:

![Capture Frame icon][]

After clicking on the Capture frame icon (the capture take a few secons), you will have access in the Debug Navigator to the OpenGL frames:

![OpenGL frames overview][]

On the device, you can see that the captured frame is displayed and paused. In the Debug Navigator, if you expand the rendering icon ![rendering-icon.png][], you will see all the OpenGL calls for the frame you’ve just captured. The Debug Navigator has a list of every draw call and state call associated with that frame (and your code call stack). The buffers associated with the frame are shown in the editor pane, and the state information is shown in the debug pane.

[![Show Debug Navigator][Show Debug Navigator 2]][Show Debug Navigator_Show Debug Navigator 2]

If you don’t see the OpenGL objects pane (the blue one), just click on the assistant icon, and select in the power bar, for instance, All GL Objects:

![Assistant power bar][]

You can right click in the Buffer pane and select Show Draw Call Highlight to see a cute green triangle on the triangle drawn by the call:

![Draw boxes][]

### 3. Inspecting OpenGL objects ###

In the assistant, you have access to the OpenGL objects like the Vertex buffers, the Index buffers, the binded (or not binded) texture, the shaders programs:

![Objects panes][]

Click on the Index buffer will give you acces to the content of the buffer:

![Index buffer][]

Click on the Programs will show you the corresponding shaders source code:

![Shaders][]

It’s a super simple and convenient way to get the source code of the shaders ”hidden” by `GLKBaseEffect`!

### 4. Improving performances ###

Just watching at the OpenGL capture frame, one can see that there is a huge obvious performance improvement to be done. In my current app, I draw 200 planes sprites with their shadow. Each plane has it own texture so the pseudo code for this captured OpenGL frame is:

glBindTexture( texture-plane-type-0) // texture for the plane of type 0
    glDrawArrays(GL_TRIANGLE_FAN, 0, 4) // draw 2 triangles (= 4 vertices in fan mode) for the sprite 
    
    glBindTexture( texture-plane-type-6)
    glDrawArrays(GL_TRIANGLE_FAN, 0, 4) 
    …
    …
    …
    …
    glBindTexture( texture-plane-type-2)
    glDrawArrays(GL_TRIANGLE_FAN, 0, 4)

In the captured frame, there is practicality one render call for each sprite, resulting for more than 6000 OpenGL calls per frame! A very first optimisation is to have as few OpenGL calls as possible. In this case, instead of binding and rendering each planes one by one, I can use a [Texture atlas][] and sprite batching to just bind one texture and only one draw call for all the planes. The resulted pseudo code will be:

glBindTexture( texture-atlas-for-planes) // texture for the plane of type 0
    glDrawElements(GL_TRIANGLE_FAN, 0, 200 * 3 * 2) // draw 200 * 6 vertices for all sprites

After having modified the code and recaptured the frame, one can see that all the planes are rendered in only one call (the green boxes show the triangle rendered by the selected OpenGL code):

[![Show Debug Navigator][Show Debug Navigator 3]][Show Debug Navigator_Show Debug Navigator 3]

The app is now at a good 60 FPS ! Xcode gives you also some advices: in that case, there is still room to add more effect and quality:

![Improvement][]

### 5. Saving and sharing captured frames ###

One final thing: you can export and save your captured OpenGL frame. You can even share it with someones that don’t have your source code, if you want an OpenGL expert to analyse your calls.

Doing this is a bit awkward, I’ve not found a better way to do this, but it works. Select an OpenGL call in the Debug Navigator, then right click on the Xcode title bar where you can see the frame capture file name (`*.gputrace`):

![Saving frame][]

You can see where Xcode temporally store the `.gputrace` file; you can then save this file in your Documents. If you want to reopen or share this file with someone, just connect a device on your Mac, double-click on the`gputrace` file and you will see this dialog box:

![Importing][]

Click on Replay trace and Xcode will launch in a self-contained mode with just this file. Very convenient!

[How to Use Xcode Frame Capture Debug Option][]

[![Author][]Issam][Author_Issam] Posted on ![Date][Author]Wednesday, October 3, 2012 at 04:10PM

One of the newest features in Xcode is the frame capture debug option for OpenGL ES. I used it a lot recently trying to figure out what Unity was doing rendering-wise on iOS, and I was thinking that other developers might not know about this and find it useful.

The only problem is that it seems to use a lot of memory, so it will only work on small Unity scenes (it crashes on bigger ones). You may have to create a new small test scene to use it. (I did not try it on an iPhone 4s or 5, I think they should have enough memory for bigger scenes.)

Once you run your application on the device, in the debug area there is a camera icon that you can use to capture a frame.

![Post1_FrameCaptureButton.png_SQUARESPACE_CACHEVERSION_1349295121786][]

For example, I had a test scene running on the iPod touch:

![Post1_TestSceneIPod.png_SQUARESPACE_CACHEVERSION_1349295589796][]

***(You can click on the following images to make them bigger)***

Doing a frame capture gave the following results in Xcode:

[![16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441][]][16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441 1]

On the left side you get all the OpenGL calls, the middle view shows the content of the currently set render buffers of the current OpenGL call selected. The other views allow you to get more details about all loaded OpenGL objects and the current OpenGL context.

[![16288177-20512403-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349295977061][]][16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441 1]

The glDrawElements(...) calls on the left side are when a mesh is rendered on the current bound buffers. For example the following image shows what happens when we select the first mesh rendered:

[![16288177-20512452-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349296187703][]][16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441 1]

On the farther right view, you can see the currently bound OpenGL objects for this call. For example we can see that a call to glUseProgram(16) happened before the glDrawElements, that's why we have program 16 as a currently bound object.

The bottom right view gives more details about objects. Right now the view is set to show all OpenGL objects, and we can see details about program 16.

On the top right view, you can double-click on the objects to get even more details about them. For example double-clicking on the texture object allows you to see all the mipmap levels of the texture and some info about it:

[![16288177-20512542-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349296769309][]][16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441 1]

Double-clicking on the program allows you to see the vertex and fragment shader GLSL code (I guess an advantage of runtime compilation :))

[![16288177-20512580-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349296905937][]][16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441 1]

Finally the bottom left view shows the current OpenGL context:

![Post1_GLContextView.png_SQUARESPACE_CACHEVERSION_1349296964940][]Also for those not familiar with Xcode, to navigate back you need to use the navigation bar at the top of each view (Took me a while to figure that out :)):

![Post1_NavigationBar.png_SQUARESPACE_CACHEVERSION_1349297091673][]

[Using Xcode OpenGL ES Frame Capture]: http://blog.manbolo.com/2012/11/20/using-xcode-opengl-es-frame-capture
[tweet]: http://twitter.com/share?url=http://blog.manbolo.com/2012/11/20/using-xcode-opengl-es-frame-capture&text=Using+Xcode+OpenGL+ES+Frame+Capture&via=ManboloGames&related=manboloGames
[Xcode OpenGL ES Frame Capture]: /images/20220731/09686ed1d1f9445282fa9b68d9e89275.png
[Instruments]: http://developer.apple.com/library/mac/#documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/Introduction/Introduction.html
[Ray Wenderlich]: http://www.raywenderlich.com/about
[OpenGL ES 2.0 for iPhone Tutorial]: http://www.raywenderlich.com/3664/opengl-es-2-0-for-iphone-tutorial
[How To Create A Simple 2D iPhone Game with OpenGL ES 2.0 and GLKit]: http://www.raywenderlich.com/9743/how-to-create-a-simple-2d-iphone-game-with-opengl-es-2-0-and-glkit-part-1
[Apple documentation about OpenGL ES Frame Capture]: http://developer.apple.com/library/ios/#documentation/ToolsLanguages/Conceptual/Xcode4UserGuide/060-Debug_Your_App/debug_app.html
[Enable capture]: /images/20220731/4777a9b8d68d4f6f9c8ed6445e53a410.png
[Show Debug Navigator]: /images/20220731/07dd1ee4f87b40a6b92d543236ba5e75.png
[Show Debug Navigator 1]: http://blog.manbolo.com/2012/11/20/screen2.png
[Debug Navigator zoomed]: /images/20220731/49a788b7d3ac4ebdbd00c1530fdd1b9a.png
[CPU vs GPU]: /images/20220731/af5e8b28a04040bba3a1bc6dc7abc580.png
[Capture Frame icon]: /images/20220731/b9254cca19454b94ba2d6d29c62283e3.png
[OpenGL frames overview]: /images/20220731/23ce63c9ea9447efacfa03a9279c7312.png
[rendering-icon.png]: /images/20220731/57ef775e5175472ebaa433b43f9b8b56.png
[Show Debug Navigator 2]: http://blog.manbolo.com/2012/11/20/screen5-mini.png
[Show Debug Navigator_Show Debug Navigator 2]: /images/20220731/4a84a0da6a754c2b97f939b3e5cf9199.png
[Assistant power bar]: /images/20220731/94ee9632ae2a45329d34c7a0d2b4d15c.png
[Draw boxes]: /images/20220731/3eb0073010684bd9bcb0339b150228d9.png
[Objects panes]: /images/20220731/9148c26020554fbba4bae22306514eef.png
[Index buffer]: /images/20220731/fc51650dbf4c47ac90a0a99666b897dc.png
[Shaders]: /images/20220731/091c4ee9b7d34ec98cc4742be9d71cff.png
[Texture atlas]: http://en.wikipedia.org/wiki/Texture_atlas
[Show Debug Navigator 3]: http://blog.manbolo.com/2012/11/20/screen6-mini.png
[Show Debug Navigator_Show Debug Navigator 3]: /images/20220731/6ecb29f814f54b1d924e8d186418ea0a.png
[Improvement]: /images/20220731/59d25c80687e4f56a01db143ec9a0aec.png
[Saving frame]: /images/20220731/2318c9a5cff542189395d306a40d0b23.png
[Importing]: /images/20220731/1061e1c7fd874c96b321925a9c5a28c3.png
[How to Use Xcode Frame Capture Debug Option]: http://www.troubleimpact.com/blog/2012/10/3/how-to-use-xcode-frame-capture-debug-option.html
[Author]: http://www.troubleimpact.com/universal/images/transparent.png
[Author_Issam]: http://www.troubleimpact.com/blog/author/issamkhalil
[Post1_FrameCaptureButton.png_SQUARESPACE_CACHEVERSION_1349295121786]: /images/20220731/92dc222e0b8f472981c6004e0c3fdea6.png
[Post1_TestSceneIPod.png_SQUARESPACE_CACHEVERSION_1349295589796]: /images/20220731/9b2575c2a09046c88ea841769b370b16.png
[16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441]: /images/20220731/919d1eb0223e47d29623689208bdfb5c.png
[16288177-20512358-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349297397441 1]: http://blog.csdn.net/s_xing/article/details/42558257
[16288177-20512403-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349295977061]: /images/20220731/4bdd5a7fa243445e96a5755ce04e8ccf.png
[16288177-20512452-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349296187703]: /images/20220731/57047500ff284048a747ee963df25a42.png
[16288177-20512542-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349296769309]: /images/20220731/c58f52ba906c436c903b6a6d952cd49d.png
[16288177-20512580-thumbnail.jpg_SQUARESPACE_CACHEVERSION_1349296905937]: /images/20220731/5d7259108b86415d917c4a48bdd182e9.png
[Post1_GLContextView.png_SQUARESPACE_CACHEVERSION_1349296964940]: /images/20220731/accf947a30044b0cab538113013eee06.png
[Post1_NavigationBar.png_SQUARESPACE_CACHEVERSION_1349297091673]: /images/20220731/9df9ba8d7ec3421a9bc68a250a8478b7.png