• 37 – Multimedia Applications
• Video Playback with One Function Call
• Fundamentals of Multimedia Programming
• Multimedia Data Formats
• Multimedia Interfaces
• Programming with MCIWnd
• The MCIWnd Window Class
• MCIWnd Functions
• MCIWnd Macros
• MCIWnd Notifications
• The Media Control Interface
• MCI Command String Syntax
• MCI Command Sets
• MCI Functions and Macros
• MCI Notifications
• Advanced Interfaces
• AVIFile and AVIStream Functions
• Custom File and Stream Handlers
• DrawDib Functions
• The Video Compression Manager
• Video Capture
• Waveform Audio Recording and Playback
• The Audio Compression Manager
• MIDI Recording and Playback
• Audio Mixers
• Miscellaneous Multimedia Services
• Summary

37 – Multimedia Applications

It has been several years since Microsoft Windows became a platform of choice for multimedia applications. Since the introduction of multimedia capabilities in Windows 3.1, hardware prices have dropped significantly. Just a few short years ago, multimedia, video, and sound required expensive workstations; now it is available on most home computers. CD-ROM drives, rapidly growing in popularity, provide the means to deliver multimedia presentations to most PCs. In short, multimedia can no longer be ignored by most programmers.

Although multimedia programming under Windows can be quite complex, applications can accomplish many things by a few simple function calls. Nothing demonstrates this better than the MCIWndCreate function, which can be used to replay videos in a single function call. Our review of Windows multimedia programming begins with a closer look at this function; for many simple applications, you may not need anything more sophisticated.

Video Playback with One Function Call

The program shown in Listing 37.1 can perhaps be viewed as the Windows multimedia equivalent of a Hello, World application. This program takes a single command-line parameter, the name of a multimedia file such as an AVI video file, and plays it back in its window. It can also be launched without a parameter; in that case, use the button controls that appear in its window (Figure 37.1) to open a file for playback.

Figure 37.1. AVI playback using MCIWndCreate.

Listing 37.1. A simple multimedia playback application.

#include <windows.h>

#include <vfw.h>

void SetClientRect(HWND hwnd, HWND hwndMCI)

{

    RECT rect;

    GetWindowRect(hwndMCI, &rect);

    AdjustWindowRectEx(&rect, GetWindowLong(hwnd, GWL_STYLE),

                       FALSE, GetWindowLong(hwnd, GWL_EXSTYLE));

    MoveWindow(hwnd, rect.left, rect.top, rect.right - rect.left,

               rect.bottom - rect.top, TRUE);

}

LRESULT CALLBACK WndProc(HWND hwnd, UINT uMsg,

                         WPARAM wParam, LPARAM lParam)

{

    switch(uMsg)

    {

        case MCIWNDM_NOTIFYPOS:

        case MCIWNDM_NOTIFYSIZE:

            SetClientRect(hwnd, (HWND)wParam);

            break;

        case WM_DESTROY:

            PostQuitMessage(0);

            break;

        default:

            return DefWindowProc(hwnd, uMsg, wParam, lParam);

    }

    return 0;

}

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,

                                     LPSTR lpCmdLine, int nCmdShow)

{

    MSG msg;

    HWND hwnd;

    WNDCLASS wndClass;

    if (hPrevInstance == NULL)

    {

        memset(&wndClass, 0, sizeof(wndClass));

        wndClass.style = CS_HREDRAW | CS_VREDRAW;

        wndClass.lpfnWndProc = WndProc;

        wndClass.hInstance = hInstance;

        wndClass.hCursor = LoadCursor(NULL, IDC_ARROW);

        wndClass.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);

        wndClass.lpszClassName = "HELLO";

        if (!RegisterClass(&wndClass)) return FALSE;

    }

    hwnd = CreateWindow("HELLO", "HELLO",

                        WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU,

                        CW_USEDEFAULT, 0, CW_USEDEFAULT, 0,

                        NULL, NULL, hInstance, NULL);

    SetClientRect(hwnd, MCIWndCreate(hwnd, hInstance, WS_VISIBLE |

                                    WS_CHILD | MCIWNDF_SHOWALL |

                                    MCIWNDF_NOTIFYSIZE |

                                    MCIWNDF_NOTIFYPOS, lpCmdLine));

    ShowWindow(hwnd, nCmdShow);

    UpdateWindow(hwnd);

    while (GetMessage(&msg, NULL, 0, 0))

        DispatchMessage(&msg);

    return msg.wParam;

}

All right, I cheated. In addition to the single function call, we also have a few lines of code handling notification messages. By responding to MCIWNDM_NOTIFYPOS and MCIWNDM_NOTIFYSIZE messages, we can ensure that the application's main window is automatically resized. That way, we end up with a window that is properly sized for video playback. Nevertheless, the application demonstrates the power of MCIWndCreate well; with that one function call, we have been able to create a full-featured video playback application.

This simple program can be compiled from the command line by typing cl hello.c user32.lib vfw32.lib.

MCIWndCreate is not the only simple function that can be used for media playback. Another such function is the Playback function; this function can be used to play waveform audio files.

Fundamentals of Multimedia Programming

Multimedia represents the operating system's ability to record and play video and sound. Multimedia programming is accomplished through a series of interfaces, or APIs that applications can utilize for this purpose.

Multimedia Data Formats

Windows recognizes three different multimedia formats: two audio formats and a video format (Figure 37.2).

Figure 37.2. Multimedia in Windows.

Waveform audio is sampled, digitized audio data. Waveform audio is typically stored in files with the .wav extension. Windows recognizes waveform audio files with mono and stereo data, a variety of sampling rates, and sampling depths. There are also several different compression methods used for the efficient storage of waveform data.

MIDI is the acronym for Musical Instrument Digital Interface. This international standard specifies a protocol for interfacing computers and electronic musical instruments. MIDI sequences are data that can be played back on MIDI-compatible instruments. MIDI data is stored under windows in files with the .mid extension. MIDI files can be played back on external devices connected to the computer, or on built-in synthesizers that support MIDI capabilities.

AVI is the Audio Video Interleaved file format. AVI files can be used to store a motion video stream and one or more audio channels. Windows recognizes video data at varying resolutions, color depths, and refresh rates. There are also several compression formats that are in wide use.

Although presently Windows does not support recording or playback of MPEG format video files, Microsoft has announced plans to do so in the future.

Windows multimedia functions can also be used for audio-CD playback. Using third-party drivers, recording and playback of other multimedia formats is also possible.

Multimedia Interfaces

Depending on your programming needs, you can choose one of three interface levels to interact with the multimedia subsystem in Windows.

The high-level interface is based on the MCIWnd window class. In the example program shown in Listing 37.1, it was an MCIWnd function that enabled us to do video playback with only a single function call.

The mid-level interface is the Media Control Interface, or MCI. MCI provides a device-independent command-message and command-string interface for the playback and recording of audio and visual data.

At the lowest level, there are several interfaces for waveform audio, AVI video, and MIDI recording and playback. Additional interfaces provide audio mixer capabilities, buffered file I/O, and joystick and timer control.

All these interfaces serve but one purpose: to provide a programming interface between end-user applications on the one hand, and drivers for multimedia hardware on the other. Through these drivers, device-independence in Windows is achieved.

Which interface should you choose for your application? That depends on the needs and requirements of your project.

Applications that require only simple playback capabilities are good candidates for using the MCIWnd services. For example, an encyclopedia application that offers video clips accompanying some articles may use an MCIWnd window for video playback.

An example for a more demanding multimedia program is an audio recording and playback application. Such a program should probably rely on the services of the Media Control Interface to implement its functionality.

A sophisticated multimedia application, such as a video capture and mixer application, would probably require using low-level video and file services.

Programming with MCIWnd

The MCIWnd window class represents the simplest, highest level multimedia programming interface in Windows. Applications that require simple playback capabilities can utilize MCIWnd windows for this purpose; such a window can be created with a single function call, as demonstrated by the program in Listing 37.1.

The MCIWnd Window Class

Windows of class MCIWnd provide a user interface that consists of up to four buttons, a trackbar, and an optional playback area (Figure 37.3).

Figure 37.3. MCIWnd Window Controls.

The Play and Stop buttons can be used to start and stop playback. Playback starts at the current position indicated by the trackbar. Special playback effects can be utilized by holding down the Shift or Control keys while clicking on the Play button; holding down the Control key results in full-screen video playback, while holding down the Shift key results in backwards play.

The Menu button can be used to invoke a popup menu. The options in this menu are specific to the type of the media file currently selected. For example, if an AVI file is loaded, the popup menu includes options to set the video playback speed, sound volume, zoom, and video configuration. Other commands enable you to copy the current data to the Windows clipboard and to open another file. Yet another menu option enables you to send an MCI command string directly to the currently active multimedia device.

The Record button can be made available for devices that can record.

The trackbar is used to display the current playback or recording position relative to the size of the file. The trackbar can also be used to move to different locations in the file during playback.

Optimal video playback performance requires that the playback window be aligned on a four-pixel boundary. Normally, Windows aligns the playback window automatically.

MCIWnd Functions

An MCIWnd window is created by a call to MCIWndCreate. A call to this function registers the MCIWnd class and creates an MCIWnd window.

In addition to specifying the handle of the parent window and an instance handle, parameters to this function also specify a set of window styles and an optional filename.

The window style settings control which elements of the MCIWnd window are visible, and how it interacts with the user on the one hand and the application code on the other. For example, by specifying the MCIWNDF_RECORD window style, you can create an MCIWnd window with a visible Record button. Specifying the MCIWNDF_NOTIFYSIZE causes the MCIWnd window to send notification messages to its parent whenever the window's size changes.

MCIWnd windows can be created as child windows or overlapped windows. If created as an overlapped window, an MCIWnd window will have a title bar with contents specified with the appropriate style settings (MCIWNDF_SHOWMODE, MCIWNDF_SHOWNAME, or MCIWNDF_SHOWPOS).

MCIWnd windows can also be created via calls to CreateWindow or CreateWindowEx. Before you can do so, however, you must call MCIWndRegisterClass. This function registers the window class specified by the constant MCIWND_WINDOW_CLASS.

There are two additional functions that use windows of class MCIWnd. The functions GetOpenFileNamePreview and GetSaveFileNamePreview enhance the standard GetOpenFileName and GetSaveFileName functions by adding a multimedia preview window to the standard file open dialog (Figure 37.4).

Figure 37.4. A file open dialog with a preview window, created using GetOpenFileNamePreview.

The program in Listing 37.2 demonstrates the use of GetOpenFileNamePreview. To compile this program, type cl ofnp.c vfw32.lib.

Listing 37.2. Using GetOpenFileNamePreview.

#include <windows.h>

#include <vfw.h>

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,

                                     LPSTR lpCmdLine, int nCmdShow)

{

    OPENFILENAME ofn;

    memset(&ofn, 0, sizeof(ofn));

    ofn.lStructSize = sizeof(ofn);

    GetOpenFileNamePreview(&ofn);

}

MCIWnd Macros

Applications can communicate with an MCIWnd window by sending messages to it using the Windows SendMessage function. A large number of helper macros exist that simplify sending most of these messages.

Through these messages, applications can control the appearance and behavior of the MCIWnd window, start and stop playback and recording, close the MCI device or file and open a new file or device for recording or playback, seek specific playback and recording positions, retrieve information on device capabilities and current settings, specify MCI device attributes, and control the MCI device.

Table 37.1 summarizes MCIWnd messages and helper macros.

Table 37.1. MCIWnd messages and macros.

Message

Macro

Description

MCIWnd Notifications

If enabled, MCIWnd windows can send notification messages to their parent windows. Specifically, five types of notification messages can be sent. All five can be enabled by specifying the MCIWNDF_NOTIFYALL window style when creating the window. Alternatively, notification messages can be enabled individually.

The MCIWNDM_NOTIFYERROR message is sent to the parent window to notify it of MCI errors. This notification can be enabled by specifying the MCIWNDF_NOTIFYERROR window style.

The MCIWNDM_NOTIFYMEDIA message notifies the parent window of any media changes that may have occurred. These messages are enabled by specifying the MCIWNDF_NOTIFYMEDIA window style.

The parent window is notified of changes in the MCIWnd window's position and size through MCIWNDM_NOTIFYPOS and MCIWNDM_NOTIFYSIZE messages. These messages are enabled by the MCIWNDF_NOTIFYPOS and MCIWNDF_NOTIFYSIZE window styles, respectively.

Finally, the parent window is notified of any operating mode changes (for example, changes from play to stop mode) by MCIWNDM_NOTIFYMODE messages. These messages are enabled by the MCIWNDF_NOTIFYMODE window style.

The Media Control Interface

The Media Control Interface provides a set of device-independent command messages and command strings for controlling multimedia devices. Command messages and command strings can be used interchangeably.

The MCI recognizes a variety of different multimedia devices. These devices are listed in Table 37.2.

Table 37.2. Multimedia devices.

Device name

Description

Both command messages and command strings can be used to control devices and to retrieve information from devices. Command messages retrieve information in the form of structures, which are easy to interpret in C programs. Command strings retrieve information in the form of strings that must be parsed and interpreted by the application.

All MCI devices support a core set of MCI commands and messages. Many devices support additional, device-specific commands.

Command strings are sent to devices using the mciSendString function. A simple use of this function is demonstrated in Listing 37.3; this application, which takes a single filename as its command-line argument, plays back a multimedia file. For example, if you specify the pathname for an AVI video file, this application will play back the video in full-screen mode. To compile this application, type cl mcistr.c user32.lib winmm.lib.

Listing 37.3. MCI playback using command strings.

#include <windows.h>

#include <stdlib.h>

#define CMDSTR "OPEN %s ALIAS MOVIE"

int WINAPI

    WinMain(HINSTANCE d1, HINSTANCE d2, LPSTR lpCmdLine, int d4)

{

    char *pBuf;

    pBuf = malloc(sizeof(CMDSTR) + strlen(lpCmdLine) - 2);

    if (!pBuf) return -1;

    wsprintf(pBuf, CMDSTR, lpCmdLine);

    mciSendString(pBuf, NULL, 0, NULL);

    free(pBuf);

    mciSendString("PLAY MOVIE WAIT", NULL, 0, NULL);

    mciSendString("CLOSE MOVIE", NULL, 0, NULL);

    return 0;

}

Command messages are sent to devices using the mciSendCommand function. This function takes several parameters, one of which is a command-specific structure. The structure may either be a general purpose one such as MCI_OPEN_PARMS or a device-specific extension to the general purpose version. Applications fill in this structure as appropriate prior to executing the call to mciSendCommand; commands that return information do so by modifying elements in this structure. The program in Listing 37.4 demonstrates simple playback using the MCI command message interface; this program can be compiled from the command line by typing cl mcimsg.c winmm.lib. Like its command string counterpart, this program also takes the name of a multimedia file on the command line and plays back that file.

Listing 37.4. MCI playback using command messages.

#include <windows.h>

#include <stdlib.h>

int WINAPI

    WinMain(HINSTANCE d1, HINSTANCE d2, LPSTR lpCmdLine, int d4)

{

    MCI_OPEN_PARMS mciOpen;

    MCI_PLAY_PARMS mciPlay;

    MCI_GENERIC_PARMS mciClose;

    mciOpen.dwCallback = 0;

    mciOpen.lpstrElementName = lpCmdLine;

    mciOpen.lpstrAlias = "MOVIE";

    mciSendCommand(0, MCI_OPEN, MCI_OPEN_ALIAS | MCI_OPEN_ELEMENT, (DWORD)&mciOp

en);

    mciPlay.dwCallback = 0;

    mciSendCommand(mciOpen.wDeviceID, MCI_PLAY, MCI_WAIT, (DWORD)&mciPlay);

    mciClose.dwCallback = 0;

    mciSendCommand(mciOpen.wDeviceID, MCI_CLOSE, 0, (DWORD)&mciClose);

    return 0;

}

MCI Command String Syntax

The generic syntax for MCI command strings is as follows:

command identifier [argument [, argument]]

The command portion specifies an MCI command such as PLAY, OPEN, or CLOSE. The identifier identifies an MCI device. This may be an MCI device name or an alias name. This identifier represents an instance of the appropriate MCI driver that was created when the device was opened.

Command arguments are used to specify flags and parameters specific to each MCI command.

For example, consider the following MCI command string:

play music from 0 to 100 wait

In this string, play is an MCI command; music is (presumably) an alias that was created when the device was opened; and from 0 to 100 wait is a series of arguments and flags applicable to the play command.

The wait flag in this command specifies that the call to mciSendString should not return before the command is finished. Normally, calls return immediately and the commands are processed in the background.

Another commonly used flag is the notify flag. Specifying this flag causes the MCI to post a multimedia notification (MM_MCINOTIFY) message to the application whenever a command is completed.

Some devices (for example, digital video devices) support the test flag. A command submitted with this flag is not executed; however, the MCI tests whether the command can be executed by the specified device and returns an error if that is not the case.

These flags can also be specified for commands submitted in the form of MCI command messages.

MCI Command Sets

MCI commands fall into different categories. These include system commands, required commands, and optional commands.

The two system commands, break and sysinfo, are recognized and processed by the MCI itself. The break command is used to set a virtual key code that aborts other MCI commands; the sysinfo command returns information on MCI services and devices.

Required commands are those that every MCI device must implement. This set includes the following five commands: capability, close, info, open, and status. The capability, info, and status commands obtain information on the status and capabilities of the device. The open and close commands are used to open or close the device.

Optional commands can be further broken down into two categories; basic commands and extended commands. Basic commands include load and save, play, record, and stop, pause and resume, seek and set, and certain forms of the status command. For most devices, it is reasonable to assume that a subset of these commands applicable to the device is supported. For example, a playback device can reasonably be expected to support at least the play and stop commands; a recording device can be expected to support record and stop.

Extended commands include several additional configuration and editing commands. Your application should not expect that any of these commands are supported; instead, it should use the capability command to find out about the available set of commands.

MCI Functions and Macros

We have already encountered two MCI functions: mciSendCommand is used to send an MCI command message, while mciSendString is used to send an MCI command string.

Another set of three functions can be used to retrieve information about an MCI device. The mciGetCreatorTask function returns the handle of the task that created a specific MCI device. The mciGetDeviceID function returns the MCI device identifier for a named device. The mciGetErrorString returns the error message string corresponding to a specific error code.

Two additional functions, mciGetYieldProc and mciSetYieldProc, can be used to yield procedure. The yield procedure is a function that the MCI calls regularly while waiting for the completion of a command that was issued with the wait flag.

The MCI also offers a series of macros that deal with time formats. Various time formats are used in MCI commands to set the recording or playback position, or to read back the current position. Positions can be expressed as time values, track positions, and so on. Time format macros are used to create position values and to retrieve individual elements of a position value. For example, MCI_HMS_HOUR retrieves the hour component of a position expressed in the form of hours, minutes, and seconds (HMS); MCI_MAKE_HMS creates a time value from parameters specifying hours, minutes, and seconds.

MCI Notifications

MCI devices can send two types of messages to the application. The MM_MCINOTIFY message is used to notify the application of the completion of a command. Parameters to this message identify the command and specify whether the command was successfully completed, or whether its execution was interrupted due to an error or some other condition.

The MM_MCISIGNAL message is used specifically in response to the extended MCI command signal. Through this command, applications can request that the MCI send an MM_MCISIGNAL message when a specific spot in the content is reached.

Advanced Interfaces

Windows offers several low-level interfaces for manipulating multimedia.

AVIFile and AVIStream Functions

AVIFile functions and macros offer low-level access to files containing RIFF (Resource Information File Format) data; examples for such files include digital video and waveform audio files.

AVIFile functions are based on the OLE Component Object Model. AVIFile functions can be used to open and close files, place files to the Windows clipboard, and obtain and manipulate file properties. AVIStream functions can be used to read or write the actual audio or video data.

Custom File and Stream Handlers

For video and audio data sources other than AVI video and waveform audio files, you can write custom file and stream handlers. Custom file and stream handlers are installable drivers that provide access to data in different sources using the OLE Component Object Model.

Custom handlers are dynamic link libraries ("inproc" OLE servers) that implement the IAVIFile and IAVIStream interfaces. These interfaces are used by the AVIFile and AVIStream family of functions.

DrawDib Functions

DrawDib functions provide high-performance capabilities for drawing device-independent bitmaps (DIBs) in 8-bit, 16-bit, 24-bit, and 32-bit graphic modes.

DrawDib functions do not rely on the GDI but write directly to video memory. The provide a variety of services ranging from image stretching, dithering, to compression and decompression of many known formats.

The Video Compression Manager

The video compression manager, or VCM, provides access to installable compressors that handle real-time video data.

Applications can use the VCM to compress and decompress video data, and handle the interaction between compressed video data, custom data, and renderers.

Video Capture

Video capture can be accomplished using the AVICap window class and a series of related functions and macros.

In the simplest scenario, applications create an AVICap window through the capCreateCaptureWindow function and send messages to this window to control capturing. A large number of macros exist that simplify the sending of messages to AVICap windows and the processing of message results.

Waveform Audio Recording and Playback

Windows offers a series of functions dealing with recording and playback of waveform audio. Simplest among these is the PlaySound function that enables you to play audio files that fit into available memory.

Other waveform audio functions can be used to control individual waveform input and output devices.

The Audio Compression Manager

Windows also provides a programming interface to the audio compression manager, or ACM. The ACM provides for the transparent compression and decompression of waveform audio data during recording and playback.

The ACM is installed as a "mapper." This means that the ACM can intercept audio recording and playback requests and decode or encode data as necessary. The ACM can also search for a waveform device or an ACM compressor or decompressor that can handle a specific format.

MIDI Recording and Playback

MIDI file playback is performed by the MCI MIDI sequencer. Low-level MIDI functions enable applications to control MIDI playback and recording, and to process MIDI data.

Audio Mixers

Audio lines can be controlled through mixer devices. Windows offers a series of functions for opening and using mixer devices.

Miscellaneous Multimedia Services

Other multimedia services include a series of functions for low-level buffered file I/O optimized for media recording and playback; file services specific to RIFF (Resource Interchange File Format); functions and notification messages for handling joystick devices; and functions to create and manage high resolution multimedia timers.

Summary

With the rapid deployment of powerful personal computers that are equipped with CD-ROM drives, multimedia has been brought to the masses. The Windows programmer can no longer ignore this area of programming; multimedia, if only in the form of sound effects or tutorial video clips, is quickly becoming a part of most new applications.

Multimedia in windows consists of the capabilities to record and play video and audio files. Windows recognizes one video format (AVI files) and two audio formats (MIDI and waveform audio).

Windows offers multimedia interfaces on three distinct levels. At the highest level is the MCIWnd window class. Through this window class, it is possible to perform video playback with a single function call. The MCIWnd API also offers a large number of macros that utilize the SendMessage function to communicate with an MCIWnd window. These windows can also send notification messages to their parent window.

The mid-level multimedia programming interface is the Media Control Interface, or MCI. MCI offers command strings and command messages for controlling multimedia devices; command strings and command messages can be used interchangeably. All MCI devices recognize a set of core commands; many devices recognize a set of additional, device-specific commands. While using the command string interface is generally simpler, for commands that retrieve information from the device it is often advantageous to use command messages instead. Unlike command strings, which return responses in the form of strings that need to be parsed an interpreted by the application, command messages return information in the form of structures.

Low-level video services include the AVIFile family of functions, interfaces to video compression and video capture, and high performance functions for drawing device-independent bitmaps. For data in nonstandard sources, custom file and stream handler drivers can be developed.

The AVIFile function family as well as custom file and stream handlers are also applicable to waveform audio. Other low-level audio services include functions for recording and playback of waveform audio and MIDI data. Additional interfaces are provided to access the audio compression manager and audio mixer devices.

Other low-level multimedia interfaces include high performance buffered file I/O, joystick control, and multimedia timers.