As I guessed, this was easy to wire up and my first pass looked like this:

WPF Example Using MediaElement

Here is a complete Visual Studio 2008 project (MediaElementPlayer) that you can build to see how it works.

Great.  Game Over, right?

Well, we watched it for a while, and found that the media player was dropping frames.  The video files would playback perfectly in Windows Media Player, but when run through this WPF app, it just didn’t work as well.  In many applications, maybe this is not a big deal, but we always want to achieve the best possible playback quality.  I tried several different attempts to optimize the application: attempting to force a specific rendering framerate, stripping down the application to just the MediaElement so that no other compositing layers or animations would tax the processor.  I tried playing lower bitrate media files, but nothing worked.  The playback still consistently dropped frames and stuttered.

I commissioned a special video file that is designed to make dropped frames and stuttering really noticeable. It features a vertical line which scrolls back and forth slowly – motion should always be perfectly smooth. You can download this video and try it with the project above.

Perhaps you can see the same behavior on your system if you build the project above.  Despite its ease of use, the MediaElement control does not allow a lot of flexibility in terms of tweaking its performance or finding out metrics on how well it is actually playing.

So I tried a different approach.  Based on some research from Jeremiah Morrill it seemed like we could use the older ActiveX Windows Media Player component to programmatically playback video inside our application. I found a tutorial and fairly quickly added the ActiveX control to playback the video files. This required building a separate forms library in order to automatically expose the necessary components as references, but it worked. This implementation seemed to harness the native Windows Media rendering pipeline and the playback performance was exactly the same as playing the video in Windows Media Player. The video was perfectly smooth again. But that wasn’t the complete solution.

The catch is that the older Windows Media wrapper was designed for Windows Forms and is only supported inside a Windows Forms Host control. This is a major problem because Windows Forms hosts can not be layered in a WPF UI layout the same way other controls can. They always show up on top, and everything else is obscured behind it. This is ultimately because of a fundamental difference between how WPF is rendered and the legacy windowing system. In this case, it meant that there was no way to add custom transport controls on top of the video. We could use the default Windows Media Player skin and transport controls, but then it gives away the secret and makes the app look thrown together. It would be much better to have the play and stop buttons match the look and feel of the rest of the application.

The final solution involved creating the transport controls in a separate transparent window and layering that on top of the video player, programmatically repositioning it automatically to create the illusion that they are part of the video player.
<Window x:Class="ActiveXMediaPlayer.TransportControlWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Background="Transparent" WindowStyle="None" AllowsTransparency="True"
ShowInTaskbar="False" ResizeMode="NoResize" SnapsToDevicePixels="True" Topmost="True">
...

This ends up requiring listening to all of the re-size and layout events from the main window and responding appropriately. It took a lot of attention to corner cases when the video player goes full screen and when the primary window loses focus, but ultimately this solution achieves the necessary effect.

You can download the improved Visual Studio project (ActiveXMediaPlayer), and compare the quality for yourself.

Many thanks to Jeremiah Morrill for his very in-depth blog that covers all aspects of video and rendering in Windows.

www.stackoverflow.com

It’s a completely free, community powered site for asking and answering questions related to software development. It was made for experts, by experts. It turned out to be an amazing problem solving resource, and shortly thereafter www.serverfault.com and www.superuser.com were opened up, and now have a huge user base. I encourage you all to take a look at the quality of questions and answers.

The founders have decided to open up to the internet community and ask for ideas to start up new sites. There are ideas for sites from mythology to raw food to industrial control systems.

I’ve put a proposal out there for a site to cater to the community of AV professionals. The concept being this is where you ask the tough questions, and help out people with tough problems. I need people to sign up and back the proposal, as well as to ask sample questions to see if the quality meets the par. This thing needs a critical mass to make it to the next stage..

A sample question could be:

When installing a BSS Soundweb in a rack, can they be stacked with no spacing? Has anyone every had heat related problems?

Or, another sample question could be:

What is a good resource for figuring out how to send wake-on-lan to various computers from an AMX controller?

A BAD question could be:

What does the blinking red light on an AMX frame mean?

So, please go to:

http://area51.stackexchange.com/proposals/8341/audio-video-control-systems

log in and post sample questions..

thanks

The new version may be installed directly over any previous version. Please let us know if you have any trouble!

An updated PrePix version 0.3.2 is available for download. New features include:

• Ability to save and reload all Screen Properties as .ppx files
• Several minor bug fixes
• Inclusion of a 6-foot scale zombie

The new version may be installed directly over the old. Please let us know if you have any trouble!

A PrePix Screenshot

Different technologies are modeled by selecting a Pixel Model, which is nothing more than a macro image representing a single pixel of the intended display technology displaying white at full brightness. Some sample Pixel Model images representing LCD and various LED technologies are included, but additional Pixel Models may be added easily by the user.

PrePix includes a graphical interface for setting up a virtual sign with parameters for Pixel Model selection, pixel dimensions, physical sign dimensions, brightness gain, RGB level adjustments, and source video selection. PrePix supports 30fps wmv video playback as well as jpg, gif and bmp images.

Source video is automatically up-sampled or down-sampled to match the pixel dimensions of the virtual video surface. Then, a Pixel Model is applied, followed by the Gain and Color Balance layers. The virtual display is viewable in a virtual 3D space.

In the 3D virtual space, the user may select from preset views including “Pixel Match,” which attempts to match large virtual LED pixels onto the PrePix user’s computer display so that the user may get a sense of the look of a virtual video surface in a real, physical space. Viewing a real 11mm LED display from 10 feet away should be comparable to viewing such a virtual display Pixel Matched to a 24” LCD display from the same distance, with the only significant difference being brightness. If this kind of modeling is deemed comparable and effective, then PrePix can be used to determine optimal viewing distances for different pixel pitches and LED technologies without requiring a lot of sample hardware.

PixelMatch close-up

An even more concrete use of PrePix is to determine the effectiveness of video or image content when displayed on certain low-resolution LED signs. The LED signs on the sides of MTA busses in NYC have pixel dimensions of 288×56. They are often sourced with video content that was obviously designed for a higher resolution display. With PrePix, a content producer can easily preview his video content on a virtual LED bus sign to check text legibility and graphical effectiveness.

288x56-pixel source image

Close-up of 3D video display model

Full view of virtual LED sign. The moire patterns is comparable to that which would be seen in a digital photo of a real LED display.

3Byte would like to develop PrePix further, depending on feedback from users. Please let us know what you think!

PrePix can be downloaded here.

PrePix System Requirements:

• Windows XP, Vista or 7
• A graphics card supporting OpenGl 3.0 and FBOs (Nearly all cards less than 18 months old.)

3Byte can be contacted at info@3-byte.com

(All videos were captured at 1000fps with a Casio Exilim EX-FS10, rendered here as 30fps video.)

When transitioning from one frame to the next, the M&M’s sign and all components of the ABC sign update all pixels on the sign at once, demonstrated here in less than one of my video capture frames, so we know it’s less than 1ms. The different components of the ABC sign seem to update at different times because of a sync issue, not an LED technology issue.

LCD displays will lock to incoming signals with a variety of timings, often anywhere from ~57Hz to ~63Hz, depending on the display. When you tell a graphics card to output at 60Hz, it is doubtful that it’s putting out a true 60.000Hz. Depending on the make of the card, the drivers installed and the phase of the moon, the actual refresh rate will vary quite a bit. LED signs tend to maintain their own clock, regardless of the video signal that might be driving it. It is likely that without a genlock signal keeping the system in sync, the source signal will not provide frames at exactly the same frame rate that the LED sign is displaying them. The more disparate the source and display refresh rates, the more dropped or doubled frames you will observe, which will be visible to the layman as stuttering.

The Reuters sign, however, DOES update one row at a time, just like an LCD. The passing of the raster takes about 14ms, as we might expect for a sign running at 60Hz:

I cannot speculate as to the implications for their sync mechanism, except to observe that the rasters of the three screen segments visible in this video seem to be in good sync, though the frames themselves do not.

When a multi-screen video system is in proper sync, each video frame begins at exactly the same time. A system that draws at 60fps will draw each frame in approximately 16.67 milliseconds. During those 16ms, an LCD display will update all of the pixels on screen from top to bottom. We will call the moving line across which the pixels are updated the Raster Line. In this slow-motion test video, you can see video frames alternating between full black and full white, updated from top to bottom. The screens are mounted in portrait orientation, which is why the updates happen right to left:

Many of the screens seem to be updating together, but some do not. This is because the system does not include dedicated hardware to ensure that the signals are in sync, so many of the displays begin their updates at different time. The system is frame-synchronized, meaning that all displays begin a raster sync within one raster-pass of each other. It just isn’t raster-synchronized.

If the displays were indeed raster-synchronized, we might represent their signals like so:

In an otherwise black video, Display 1 flashes a single frame (Frame 1 / F1) of white, followed by a flash of white on Display 2 in Frame 2. In slow-motion, we would see the raster line, represented here as diagonal lines, move across bother displays in sync, like two Tangueros walking together across a dance floor. The red line represents the particular pass of the raster line at which both displays transition from Frame 1 to Frame 2. In all of these illustrations, the red line indicates a pass of the raster line, or a switch between frames, that in a frame-synchronized system would occur at exactly the same time.

It is important to keep in mind that in this system, video is provided at 30fps, so each frame of video is essentially drawn twice in two consecutive passes of the raster line. This cannot be seen on screen as no change occurs in any of the pixels, but we can see it in our illustration as the diagonal line separating F1a and F1b, the two rendered passes of video Frame 1.

In our system, of course, the raster lines are not synchronized between displays. So even when we intend for both displays to flash white at the same time, it is possible that one display might begin a raster pass at the very end of another’s pass of the same frame:

We can certainly hope that the raster lines of any two average displays are nearly synchronized, but in observance of Murphey’s Law, we must always assume the worst case, as indicated above in Displays 1 and 2. Here, we can see that although the frames might be in sync, with all rasters commencing within 17ms of each other, we will expect to see the raster on Display 2 commence just at the end of that of Display 1. This kind of behavior can be seen in our test video on the third and fourth columns, with the raster seeming to pass smoothly from one display to the other. In truth, any case in which two rasters start more than 16.67ms apart from each other demonstrates an imperfect frame sync, but for simplicity we will just say that the worst case in one in which they commence 17ms apart, as illustrated above in Displays 1 and 2.

So, what might this look like in a case where we see flashes in two consecutive video frames on two separate displays in a worst-case scenario?

In the case of Displays 1 and 4, we have a point in time, Time X, at which both displays are entirely black. In the case of Displays 2 and 3, at Time X we see both displays entirely white. We still consider them to be in sync, and we should not consider these anomalies to indicate a failure of the synchronization mechanism.

There is a minor issue in the test video that does bear mentioning. The flashes move through four rows of white in each column. There are extremely brief moments during which you might notice a touch of white that is visible in the first and third row. This should be obvious after repeated viewing. I leave it as an exercise for the reader to demonstrate why, even in a worst case scenario, this should not be observed if the frame synchronization mechanism is working properly. (<sarcasm>Yeah, right.</sarcasm>) So why am I not concerned? Because it is the nature of LCD display pixels to take some time to switch from full white to full black. Typical response times for the particular displays in this system are specified as 9ms, which means that after the raster line passes and updates a pixel from black to white, it may take an average of 9ms (more than half a raster pass) for that pixel to fully change. I say “Average” because white-to-black and black-to-white transitions are usually slightly different, and the spec will mention only the average of the two. If the response time was an ideal zero ms, our raster line would be crisp and clear in our slow-motion capture, but in reality it is not. The raster line is blurry because after it passes, the pixels take time to change between black and white. We can expect that some pixels might remain white for a brief time after we expect them to go dark, resulting in this subtle observable discrepancy.

What does all this mean? It means that in a slow-motion test video of 24 synchronized displays, we observe nothing to suggest that the synchronization mechanism isn’t performing as well as we could hope for. To the viewer, the synchronization is true, and we deem the project a success.

To test our synchronization, we loaded up a video file that would make the quality of the sync really obvious. Breaking the video vertically into four quarters, on each frame of the video we flash a white rectangle in one of the quarter regions. Like so:

Really, it works better with some good PsyTrance. But what is actually happening? Whatever it is, it’s happening too fast to determine with the naked eye. So I picked up a Casio Exilim EX-FS10 and shot the system at 1000fps:

Far more interesting. The resolution of the video is not great, but it shows us what we need to know.

First of all, you will notice the scrolling of the video from right to left. The screens are actually mounted in portrait orientation, to the motion that you see is actual a top-to-bottom scroll as far as the display is concerned. These LCD displays refresh the pixels on screen 60 times every second, but they don’t all update at the same time. The display updates the pixels line by line from top to bottom, each line updating left to right. This makes sense when we consider that analog and digital video signals order pixel data in this way, just like reading a page of a book. At regular speed, a screen refresh seems instantaneous, but at high speed, we can see the way the screen transitions from black to white one line at a time. This scrolling line across which the pixels are updated we might call the Raster, or the Raster Position.

In this system, the timing of each display’s Raster Line is determined by the graphics card in the computer. Whenever the card says “start drawing the next frame… NOW!” the monitor locks in with that signal and starts the Raster Line at the top of the screen. Had we G-Sync cards for this system, we could tell the graphics cards to chant their “NOW” mantras in unison, and in slow motion we would see the Raster Lines of all the displays being drawn in perfect synchronization. As you can see in the video above, this is not the case for our system, where the lines are updated on different displays at slightly different times. This difference between displays is so subtle that it is never noticed by a viewer. The question is, are the correct frames being drawn on each pass of the Raster?

This system supports source video playback at 30fps, but the displays update at 60fps. Each source video frame is doubled on screen, so a single frame of white flash in our source video will be drawn white in two consecutive passes of the Raster Line on the display. In the slow-motion video, we see the raster line update each screen, then a pause while the subsequent Raster pass draws another frame of white (no change) before moving on the to next source video frame.

If you look at the third and fourth columns of displays, you will see that the Raster seems almost to move straight across from the fourth to the third as it updates the two columns together. Of course this is only an illusion, as the Raster does not sync between frames. What we are actually seeing is one display in column three that seems to be lagging behind column four by almost a full 17ms Raster pass. (I say 17ms because that is just about the amount of time it takes to refresh a display at 60fps.) This is not ideal, but in a system with no dedicated sync hardware, it is not surprising, and not a deal-breaker. It means that at 30fps, these screens are within a half-frame of perfect sync, which is nearly undetectable to the eye.

In Part II, I provide an analysis of the best possible sync performance for a network-synchronized video system. I  explain why the 17ms discrepancy in the video above falls within the tolerances of this system. We are quite pleased with the performance of our synchronization mechanism, and believe that it rivals or surpasses that of other industry-standard network-sync video systems. Next chance I get, I’ll run a similar test on a Dataton Watchout system and let you all know how it goes. Stay tuned.

I remember a few years ago on a certain project in Texas where the AV guys (I was the software consultant) absolutely refused to consider using an off the shelf Dell computer and a custom video playback app to run video to the screens. The options given to us were an Alcorn DVM-HD or a GVG Turbo. I mentioned to them that both of these systems were actually embedded PC’s with a custom PCI-E video output card (one ran XP pro embedded and the other some sort of Linux distro, or maybe even DOS- who knows). However, this didn’t seem to matter. it was all about how the non-computer felt and looked in the racks (and, they even threw the old argument- there isn’t enough rackspace). Come on, really? Maybe they didn’t trust my custom software solution, but Josh and I were building the master show control system running on multiple PC’s and servers, so I don’t see the logic.

Anyways, that project had a truckload of problems concerning video playback. My argument was that it’s the idiosyncrasies that end up dictating how these systems are run, and by holistically controlling one of the more important parts of the project (the video playback engine) we controlled the idiosyncrasies. After the system finally ended up stabilizing by buying twice as many devices (because the dual channel capability actually didn’t work too well in practice) the main issue was that the 74GB HDD size was way too small, but it was the biggest WD raptor drive available. oh well.

So, back to mission critical stuff running on PC’s. Here is a story about BAE systems installing Win XP & 2k as a mission critical command and control system in the Royal Navy’s Trafalgar and Vanguard class nuclear subs:

BAE Win XP and Win2k on subs

How about that? If we dial it back to the level of AV systems, consider personal mission critical stuff. Like your communications device. I bought an iPhone last week. It’s basically a mini computer. it’s now loaded with over 40 applications, and who knows who wrote these things? I’m not afraid this thing is going to “crash” if I dial 911, why would you be afraid if a video screen somewhere glitches for an hour or two? It’s obviously different if you are in a theatre situation, but that is what backup hardware, and, ahem, professional quality code and professional project management, is for.

PS, there is a whole slew of pictures on the internet of random massive LED screens in times square and other places showing windows error screens that stay up for DAYS. That’s just pathetic and reeks of poor planning. Why would you drop 1+ mil on that thing and not have a plan to service & monitor it?

here are some fun pics