Very Fast Screen Capture With Windows 8, C# and DirectX11

Lately I have experienced problems while using my selfmade Ambilight.
This Ambilight consists of LED-strips that are clamped to the back of my monitor, an Arduino as control unit and finally a tool running on the OS to measure and transmit the colors that currently are displayed on the screen. The program ran more or less OK on Windows 7 – just for your information, until today I have been using the DirectX 9 approach which you can read about here: http://www.codeproject.com/Articles/5051/
Various-methods-for-capturing-the-screen
. Whenever the program was slowing down my computer too much I would disable DWM (don’t know why it performed better without the new DWM, guess it was just not so well integrated at that time).

Now, after moving city, I installed the Ambilight to my computer monitor again. I didn’t only change residence, but also OS. I have been shocked to find out that with Windows 8 DWM can’t be turned off anymore. So lately my Ambilight became more a pain in the ass then it was entertaining really. I spent the last two days surfing the web and searching for a new/better way of capturing images of the desktop (and maybe even fullscreen games etc).

Eventually I discovered some articles about a new DXGI version (1.2) that came out with Windows 8 and which introduced Desktop Duplication.

My current program was written in C# and used the SlimDX wrapper to access the DirectX API. SlimDX sadly does not support version 1.2 of the DXG Infrastructure yet, that’s why I moved to SharpDX. Pay attention to reference the libraries from the “Win8Desktop-net40” folder, otherwise you won’t have the new DXGI interface.

Below you will find a little test program that I wrote to test performance of this new approach:

Update:

I will just summarize the main points that arose from the comments below (and some others that I figured out during the last few days):

  • Although MSDN states that at least Windows 7 with Platform Update is required to use the DesktopDuplication, this is not true. The Platform Update will just follow the interface convention of DXGI 1.2 but won’t implement the newly added methods. (instead they will just fail with error code E_NOTIMPL.
  • I haven’t found a very efficient way to save the SharpDX.DataStream yet. I mixed up some code from Stackoverflow that will help you to transform the stream into a Bitmap object which can be saved easily.
  • Fullscreen DirectX applications can not be captured without source code control. If the SwapChain hasn’t been created with the right flags, there is (at least without any system-hacking-skills) no way to duplicate the screen. You can find more information about that on the SwapChain Flags MSDN page. Now as I see it you will either have to use IDXGIFactory2::CreateSwapChainForHwnd, IDXGIFactory2::CreateSwapChainForCoreWindow or IDXGIFactory2::CreateSwapChainForComposition without the DXGI_SWAP_CHAIN_FLAG_DISPLAY_ONLY to allow other applications to access the SwapChain’s backbuffer.

38 thoughts on “Very Fast Screen Capture With Windows 8, C# and DirectX11

  1. Thanks for your comment!

    I found this article – as most of the MSDN stuff – quite misleading …
    Although Windows 7 with Platform Update is listed under “Minimum supported client” it sadly is not.
    A few lines above it says

    Platform Update for Windows 7: On Windows 7 or Windows Server 2008 R2 with the Platform Update for Windows 7 installed, DuplicateOutput fails with E_NOTIMPL.

    and on the platform update page they say:

    These classes of features are unavailable on Platform Update for Windows 7:
    Stereo
    Swapchains not targeting Hwnds
    Occlusion status notifications
    Desktop duplication
    NT Handle resources

    Really sad 🙁 – but eventually (and at least) a clear statement.

  2. Berney Villers says:

    Sadly you are correct. I only glanced at the bottom to check compatibility.

    Can you say how to get this into a bitmap? I previously used SlimDX to get frontbuffer, and there was a Surface.ToFile and also the ability to create a Direct2D Bitmap from a stream.

    I don’t see either of these available within DXGI / SharpDX.

  3. I too was looking for these methods. I only wanted to process the data pixel by pixel though and then I didn’t bother anymore.

    I took a code snippet from this article http://stackoverflow.com/questions/6782489/create-bitmap-from-byte-array-of-pixel-data and modified it to take the DataStream that results from the Surface.Map method and returns a Bitmap though. Hopefully it helps.

  4. Berney Villers says:

    Thanks!

    May I ask what SharpDX assemblies you are referencing? I am getting back all zeros when I loop through and print out the datastream with readbyte.

    I have references to Direct3D11 and DXGI from win8-desktop-net40 version 2.5.0.0. Perhaps I am missing a needed reference.

  5. Oh, I forgot to mention, that the first image captured always seems to be black.
    I don’t know whether it is caused by my implementation or it is an undocumented “feature” …
    So the second time you call AcquireNextFrame it will return a valid image.
    (I am using the same assemblies, the behaviour should be equal for you then)

  6. Berney Villers says:

    Indeed, the first image is all black. The remainder are unfortunately all garbled using the stackoverflow code.

    I am yet to figure out how to get this Texture2D converted properly to a GDI compatible Bitmap. I suspect that Direct3D or WIC has functions for this.

    Are you getting proper images on your end? Perhaps I muddled something up when I converted your code to VB.Net.

    This is what I have at the moment.

  7. Berney Villers says:

    I apologize for the mess above. I don’t know how you made the nice collapsible code block.

  8. No problem, I fixed it :-).
    Don’t know whether this works for everyone, but normally you can use a [[code]] block.

    I can’t find differences from my implementation. And as much as I have been testing it worked fine.
    You say that the image is garbled? What I could imagine is that the order of rgb values is wrong.
    I already wondered because it says “PixelFormat.Format32bppArgb” and the SharpDX.DataStream normally delivers Bgra.
    Though I can’t imagine why the order would be reversed on your end, you could try to change the sorting of colours written to the rgbValues-array.

  9. Berney Villers says:

    I think there are some errors in that image conversion code. I’m still not quite able to get an accurate screenshot.

    One thing that the SharpDX guys pointed out to me is that stride is already in bytes, so no need to multiply that * 4.

    Unfortunately, after correcting that, I still get a garbled image, so something else is amiss.

  10. Berney Villers says:

    I tried this code below which surely puts the bytes in the correct order. I think the issue is not related to image conversion, but rather the bytes I am getting out of the sample desktop duplication code.

    Would you be willing to share a little sample project? I converted this to VB.Net. I am not a C# guru. When I attempted to place you code into a test C# solution it was complaining to me that it couldn’t infer types for the QueryInterface portions of the code.

  11. Berney Villers says:

    🙁

    Here is the code I meant to paste above:

  12. Eric says:

    The latest version of SharpDX requires an update to the code posted here. These are the corrected line:

  13. Thanks for your comment Eric.
    I have merged your comments and put the code into a sourcecode-block. (I hope I recovered it correctly, just leave me a message if I messed it up)

    Sorry for my late answer Berney, as it seems you found an answer to your question already. I haven’t looked at the differences between the code snippets yet. The code that I posted works for me without any modifications 😕 .

  14. Eric says:

    Good job Flo, thanks! I’ve been checking out the methods described here and I am impressed. Haven’t really been getting the performance I want though .. I too am trying this for custom ambilight.

    Windows 8’s DWM requirement has had quite the impact on my Winforms Screen capture util and I am looking for new ways to replicate the sluggish behavior I am getting now.

    Are you willing to share some more code from your solution? What are the FPS you are getting?

    My current method (BitBlt) is the fastest, even faster than some of the excellent d3d (9,10 and 11) hooks I have found here.

    http://stackoverflow.com/questions/1994676/hooking-directx-endscene-from-an-injected-dll

    The trick, that even makes BitBlt fast using DWM on Windows 8, is to always operate on target window handles instead of the entire screen.

  15. Thank you!
    I just uploaded the sourcecode to the tool that I am using to control my ambilight.
    http://www.floschnell.de/wp-content/uploads/2014/10/AmbiMon.zip
    Most of the code won’t be too interesting, look out for the MeasureColors and GetPixel methods in the Program.cs file.
    I am getting up to 60 and more fps. Frequency changes with how often the OS needs to redraw the screen surface (so for instance moving the mouse very quickly will deliver high amount of fps).
    The fact that BitBlt is faster for you sounds odd … Maybe you’re reinitializing some pieces of code too often?
    I am looking forward to hear if you have found the cause for this!

  16. Randy says:

    One thing I noticed in the performance test code is that the frame count is incremented before the actual capture (i++), but the catch block doesn’t decrement the frame count — leading to artificially high frame rates. A simple “i–;” before the “continue;” is all that’s needed.
    That said, a huge “thank you” for doing the grunt work to research this and also being kind enough to publish your findings!

  17. @roger: I don’t have one. But I’m totally sure that bitblt is much slower if you need to retrieve many pixels from one frame. Maybe performance is comparable if it is only a handful of pixels that you need to get.

    @Randy: thank you, I missed that. Your adjustments have been added 🙂

  18. xxxo says:

    Hi !

    I managed to make it work but I have an issue with the console : it’s all black, its content is not captured…

    Any idea on how to make it capture the actual content of the console ?

  19. Greg says:

    Is it possible to render custom image/surface onto the display but Not have it show up on images captured using Desktop Duplication API?

  20. It might, but I don’t know an answer to this.
    Well you can render Direct3D stuff onto the screen and if you are configuring the pipeline with the right arguments, then Desktop Duplication will not be able to grab the contents.
    But I think it will be rendered as a black overlay and probably hide some other UI elements on the screen.

  21. Paul says:

    Just spent the last 3-4 days trying to get a “screen-capture” of my rendering for the purposes of saving images. I’m using c# and SharpDX. Used to use XNA and it was so simple to do there… After reading this I was able to finally make some strides. One MAJOR hurtle that took me awhile to figure out why it wasn’t working was I had my backbuffer created with Multi-Sampling set. After another day of banging my head into my keyboard and multiple searches in google I finally foudn a very simple solution for creating an image of my anti-aliased backbuffer.

    (lets see if I use this code tag properly…)

  22. Thank you Paul for sharing your information and code here!
    Glad we could help you with our idea-collection as well.
    If I understand correctly, you put this code directly into your rendering code, right?
    As far as I can see you’re first rendering your scene to the backbuffer, which you then will copy into an image texture?
    (I’m just asking because I’m still interested in whether it is possible to grab the DirectX surface of a different application)

  23. David says:

    Hello Florian

    Firstly I’d like to thank you for sharing this fast method of capturing the screen.
    It is really fast but the only bottleneck I had was your way of converting the DataStream to a Bitmap.
    After playing around i found this way of copying the DataStream to the BitmapData:

    var b = new Bitmap(width, height, PixelFormat.Format32bppArgb);
    var BoundsRect = new System.Drawing.Rectangle(0, 0, width, height);
    BitmapData data = b.LockBits(BoundsRect, ImageLockMode.WriteOnly, b.PixelFormat);

    stream.Read(data.Scan0, 0, data.Stride * b.Height);

    b.UnlockBits(data);

    This method yields a conversion time of about 5 ms for me.

  24. TheJMan says:

    Thanks for the quick response! I am now trying out the code, but do not have my arduino attached. Would you also have the arduino code available? If not, I can deduce it from the C# code. Basically the arduino has to read the serial data, where the first byte is the nr for the led followed by rgb right?

    By far the best working and fast solution I have seen up till now. Can’t wait to try it with my 100 led setup 😀

  25. Jerome says:

    Hi Florian,

    Great article ! It helps a lot. I’m working on a DIY Amibilight too. I started to experiment with the GDI option. It could have been acceptable if the Graphics.CopyFromScreen() method did not micro-freeze the whole UI. It is almost unnoticable, but when I move an other Window, I can see it is not as smooth as usual.

    So I started looking for an other way to do it, with DirectX and I luckily landed here 🙂 I’ve followed your exemple, with light changes (from here : https://github.com/Lakritzator/SharpDX-Samples/blob/master/WindowsDesktop/Direct3D11.1/ScreenCapture/Program.cs ).

    It is definitely faster than the GDI option without freezing the UI. But I have a new problem : when I move the mouse pointer on the captured screen, it captures black pictures only (no trouble when the mouse is on my secondary screen). When I stop moving the mouse, it works again. I have also noticed that when I dragg a Window with the mouse, it keeps working. The issue occurs only when the mouse pointer is free (= not attached to a Windows) and moving.

    Did you meet the same trouble ?

    Thanks for this article 🙂

  26. TheJMan says:

    Hello Flo,

    Do you also have the arduino code available? I am having trouble getting it to work without it…

    Thanks !

  27. TheChoclateMan says:

    Hello Flo,

    I also like to take a look at your Arduino code, I have trouble with the serial timing of 240 leds. I group them by six so I only need to control 40.

    It works but not flawlessly maybe your code could help a bit.

    Thanks in advance!

  28. Albert says:

    I’ve been trying to make this code work (and similar versions, like the one on github/SharpDX) with no success. I keep getting this:
    HRESULT: [0x887A0004], Module: [SharpDX.DXGI], ApiCode: [DXGI_ERROR_UNSUPPORTED/Unsupported], Message: The specified device interface or feature level is not supported on this system.

    This is when trying to duplicate output. I’m using Sharp 3 alpha, as the stable version seemed to miss some classes (e.g. Output1).
    I have a geforce 970m on a win8 laptop (which also has an intel 4600).
    Any hints as to why would this be happening?

  29. Okay, so the first thing that comes to my mind is that you graphics card does not support directx 11. But I checked and intel 4600 normally should. Have you got DirectX 11 installed?

  30. @TheJMan and TheChoclateMan: I was looking for my arduino code, but I just can’t find it anymore 🙁 . I hope to stumble over it in the future and then I will post it right here.

  31. Albert says:

    yes, dx11 is installed, and both gpus support that version. I wonder if it is related to the dual card setup or the version of SharpDX (alpha3)

  32. booler0 says:

    thank you, it is indeed very fast..

    you need aforge libraries for this class to work

    using SharpDX.DXGI;
    using System;
    using System.Collections.Concurrent;
    using System.Diagnostics;
    using System.Drawing;
    using System.Drawing.Imaging;
    using System.Runtime.InteropServices;
    using System.Threading.Tasks;

    namespace ScreenCaptureServer
    {
    public class ScreenRecorder
    {
    BlockingCollection bitmapsToRender = new BlockingCollection(10);
    BlockingCollection freeBitmaps = new BlockingCollection(10);
    bool keepRendering = false;
    bool keepRecording = false;
    Task tsk;
    string outputFile;
    int fps;

    public int RecordingFPS { get; private set; }
    public bool IsRecording { get; private set; }

    public ScreenRecorder(string outputAviFile = “test.avi”, int FPS = 30, string codec = “X264”)
    {
    this.outputFile = outputAviFile;
    fps = FPS;
    }

    public void StartRecording()
    {
    keepRecording = true;
    tsk = new Task(new Action(StartCapture));
    tsk.Start();
    IsRecording = true;
    }

    public void StopRecording()
    {
    if (IsRecording)
    {
    keepRecording = false;
    }
    }

    void StartCapture()
    {

    uint numAdapter = 0; // # of graphics card adapter
    uint numOutput = 0; // # of output device (i.e. monitor)

    // create device and factory
    SharpDX.Direct3D11.Device device = new SharpDX.Direct3D11.Device(SharpDX.Direct3D.DriverType.Hardware);
    Factory1 factory = new Factory1();

    // creating CPU-accessible texture resource
    SharpDX.Direct3D11.Texture2DDescription texdes = new SharpDX.Direct3D11.Texture2DDescription();
    texdes.CpuAccessFlags = SharpDX.Direct3D11.CpuAccessFlags.Read;
    texdes.BindFlags = SharpDX.Direct3D11.BindFlags.None;
    texdes.Format = Format.B8G8R8A8_UNorm;
    texdes.Height = factory.Adapters1[numAdapter].Outputs[numOutput].Description.DesktopBounds.Bottom;
    texdes.Width = factory.Adapters1[numAdapter].Outputs[numOutput].Description.DesktopBounds.Right;
    texdes.OptionFlags = SharpDX.Direct3D11.ResourceOptionFlags.None;
    texdes.MipLevels = 1;
    texdes.ArraySize = 1;
    texdes.SampleDescription.Count = 1;
    texdes.SampleDescription.Quality = 0;
    texdes.Usage = SharpDX.Direct3D11.ResourceUsage.Staging;
    SharpDX.Direct3D11.Texture2D screenTexture = new SharpDX.Direct3D11.Texture2D(device, texdes);

    // duplicate output stuff
    Output1 output = new Output1(factory.Adapters1[numAdapter].Outputs[numOutput].NativePointer);
    OutputDuplication duplicatedOutput = output.DuplicateOutput(device);
    Resource screenResource = null;
    SharpDX.DataStream dataStream;
    Surface screenSurface;

    int i = 0;
    Stopwatch sw = new Stopwatch();
    sw.Start();

    keepRendering = true;
    Task render = new Task(new Action(Render));
    render.Start();

    for (int count = 0; count < freeBitmaps.BoundedCapacity – 1; count++)
    {
    freeBitmaps.Add(new Bitmap(texdes.Width, texdes.Height, PixelFormat.Format32bppRgb));
    }

    while (keepRecording)
    {
    i++;
    // try to get duplicated frame within given time
    try
    {
    OutputDuplicateFrameInformation duplicateFrameInformation;
    duplicatedOutput.AcquireNextFrame(1000, out duplicateFrameInformation, out screenResource);

    }
    catch (SharpDX.SharpDXException e)
    {
    if (e.ResultCode.Code == SharpDX.DXGI.ResultCode.WaitTimeout.Result.Code)
    {
    // this has not been a successful capture
    // thanks @Randy
    i–;

    // keep retrying
    continue;
    }
    else
    {
    throw e;
    }
    }

    // copy resource into memory that can be accessed by the CPU
    device.ImmediateContext.CopyResource(screenResource.QueryInterface(), screenTexture);
    // cast from texture to surface, so we can access its bytes
    screenSurface = screenTexture.QueryInterface();

    // map the resource to access it
    screenSurface.Map(MapFlags.Read, out dataStream);

    // seek within the stream and read one byte
    dataStream.Position = 0;

    var bmp = freeBitmaps.Take();
    var BoundsRect = new Rectangle(0, 0, bmp.Width, bmp.Height);
    BitmapData bmpData = bmp.LockBits(BoundsRect, ImageLockMode.WriteOnly, bmp.PixelFormat);
    int bytes = bmpData.Stride * bmp.Height;

    var rgbValues = new byte[bytes];

    int a = dataStream.Read(rgbValues, 0, bytes);
    Marshal.Copy(rgbValues, 0, bmpData.Scan0, bytes);

    bmp.UnlockBits(bmpData);
    bitmapsToRender.Add(bmp);

    // free resources
    dataStream.Close();
    screenSurface.Unmap();
    screenSurface.Dispose();
    screenResource.Dispose();
    duplicatedOutput.ReleaseFrame();

    // print how many frames we could process within the last second
    // note that this also depends on how often windows will >need< to redraw the interface
    if (sw.ElapsedMilliseconds >= 1000)
    {
    RecordingFPS = i;
    sw.Reset();
    sw.Start();
    i = 0;
    }
    }

    keepRendering = false;
    }

    void Render()
    {
    int count = 0;
    AForge.Video.VFW.AVIWriter aviWriter = new AForge.Video.VFW.AVIWriter(“X264”);

    bool isFirst = true;
    while (keepRendering)
    {
    Bitmap bmp = bitmapsToRender.Take();

    if (isFirst)
    {
    aviWriter.Open(outputFile, bmp.Width, bmp.Height);
    aviWriter.FrameRate = 30;
    aviWriter.Quality = 100;
    isFirst = false;
    }

    aviWriter.AddFrame(bmp);

    freeBitmaps.Add(bmp);
    count++;
    }
    aviWriter.Close();
    }

    }
    }

Leave a Reply

Your email address will not be published. Required fields are marked *