Delivering forty channel audio using QuickTime APIs to adjust movie files.

1. Movie qtMovie = [movie quickTimeMovie];
2. Track audioTrack = GetMovieIndTrack(qtMovie,1);
4. AudioChannelLayout* trackChannelLayout = NULL;
5. OSStatus err = noErr;
6. UInt32 trackChannelLayoutSize;
9. // Allocate a layout of the required size
10. trackChannelLayoutSize = fieldOffset(AudioChannelLayout, mChannelDescriptions[2]);
11. trackChannelLayout = (AudioChannelLayout*)calloc(1, trackChannelLayoutSize);
12. trackChannelLayout->mChannelLayoutTag = kAudioChannelLayoutTag_UseChannelDescriptions;
13. trackChannelLayout->mNumberChannelDescriptions = 2;
14. // Adjust the channel Assignment so that for an index we get a pair of channel
15. // allocations. The index starts at 1
16. // index channelA channelB
17. // -----------------------------
18. // 1 discreet0 discreet1
19. // 2 discreet2 discreet3
20. // 3 discreet4 discreet5
21. // 4 discreet6 discreet7
22. // ... and so on
23. channelAssignment = (channelAssignment-1)*2;
24. #warning This will break if Apple change the CoreAudio's Channel Layout internals
25. trackChannelLayout->mChannelDescriptions[0].mChannelLabel = (1L<<16) | channelAssignment;
26. trackChannelLayout->mChannelDescriptions[1].mChannelLabel = (1L<<16) | (channelAssignment + 1);
29. // Set the track layout
30. err = QTSetTrackProperty(audioTrack, kQTPropertyClass_Audio, kQTAudioPropertyID_ChannelLayout, trackChannelLayoutSize, trackChannelLayout);
31. if (err != noErr) {
32. NSLog(@"**Error** QuickTime SetPropertyError: %i", err);
33. return 1;
34. }

Download the full source code.

Tags: QuickTime, Cocoa, Core Audio

|

Interactive QuickTime is pretty much dead and the QuickTime (.mov) file format is dying.

A short while ago, whilst working on Electric December, I spent some more time working on server generated, interactive video presentation. This was an exact repeat of the previous year's Electric December development and once again I abandoned the work after a couple of weeks.

Yes, I know this type of thing can be done easily enough in Flash but I wanted to do it in QuickTime for a few reasons:

• We want to keep the video content in MPEG4 data files because we don't want to encode very large amounts of our content in a proprietry codec.
• In my opinion, the pereformance of MPEG4 is better than the Flash On2 codec.
• We need to distribute our content in a format that can be played on mobile devices such as iPods, PSPs and of course the iPhone.

In the Applesphere it appears that the QuickTime file format has been beaten down by MPEG4. The Apple TV doesn't appear to support QuickTime files and neither does the iPod. If the ipod doesn't support quicktime files then it is very unlikely the iPhone will. Interactivity and the other advanced features of MPEG4 are not supported in QuickTime and MPEG4 is seen as a linear audio and video format.

Interactivity in QuickTime has been neglected and become an embarrassment for Apple. QuickTime VR is hardly mentioned these days. Have you tried opening a .m4a file with chapter artwork and links in QuickTime Player? How ugly is the result? The chapter implementation in the QuickTime UI is neat and tidy but why draw text using Times, in 'link blue' across the artwork?

Apple recently disabled QuickTime's default ability to play Flash movies embedded inside a movie. Security issues with the existing code led Apple to turn this option off, without warning content producers this was going to happen. Flash playback in QuickTime couldn't have been a high priority. The user can re-activate the functionality, but the settings are hidden many levels down in nested preference panels, and not something you would expect an end user to perform in order to view content.

QuickTime's XML parsing and resource loading performance makes using techniques such as SMIL or wired, umbrella movies very difficult to deploy. How long can it take to display a few, small, simple media resources such as text and still images?

There are so many differences between the implementations of features, that haven't changed, in versions 6 and 7 of QuickTime? A content developer can't distinguish if something is working or not. Well, apart from text rendering which is not great in either. Why does QuickTime 7 install two different versions of the plug-in? One, a Coca/QTKit Webkit plug-in used by Safari, and the other, a Netscape style plug-in that all the other Macintosh browsers use. It would be quite nice if these two plug-ins displayed content and behaved identically. Based on content from Apple's own site, the Quicktime plug-in is used simply to provide basic, interactive playback of linear MPEG4 video controlled by Javascript. The interface is provided by the surrounding HTML.

It appears all the graphics & media engineers at Apple are working on technologies such as such as CoreAnimation, CoreVideo & Quartz Composer. All the advanced media technology work is done on the Mac OS X platform and not QuickTime! This is all great stuff and I love working in CoreVideo and CoreAnimation. They produce spectacular results and I amaze myself every time I click Build & Run. This is nothing we can deploy effectively to the public at large, for which the MS Windows using majority, the QuickTime plug-in is the only bridge there is. QuickTime will become a media generation framework to support Apple's pro & iLife applications, along with third party developer's desktop applications. The browser plug-in will just handle playback of MPEG4 data.

I guess what I'm trying to say is don't waste any more time trying to use QuickTime for interactive work, use it for what it's best at: playback of MPEG4 content and leave the interactivity & interface to Javascript & HTML.

Tags: QuickTime, web development

Generating still JPEG images from QuickTime movies using Java running on the server.

1. import quicktime.*;
2. import quicktime.app.*;
3. import quicktime.app.players.*;
4. import quicktime.app.display.*;
5. import quicktime.io.*;
6. import quicktime.std.*;
7. import quicktime.std.movies.*;
8. import quicktime.qd.*;
9. import quicktime.std.movies.media.*;
11. public class GetImage {
12. public GetImage() {
13. }
15. public void getImageFromMovie(String url, String offsetTime) {
17. try {
18. QTSession.open();
19. DataRef ref = new DataRef(url);
20. System.out.println("DataRef: " + ref);
21. if (ref == null) return;
23. Movie theMovie = Movie.fromDataRef(ref, 0);
24. System.out.println(theMovie.getBounds());
26. int theTime = Integer.parseInt(offsetTime) * theMovie.getTimeScale();
27. System.out.println("theTime: " + theTime);
28. Pict pictImage = theMovie.getPict(theTime);
30. System.out.println("PICT: " + pictImage);
32. // Optionally you could just use the poster image if it has been set by the person encoding the video
33. //pictImage = theMovie.getPosterPict();
35. // Now encode the Macintosh PICT as a JPEG or PNG or ....
36. quicktime.std.image.GraphicsExporter graphicsExporter = new quicktime.std.image.GraphicsExporter(StdQTConstants.kQTFileTypeJPEG);
37. graphicsExporter.setInputPicture(pictImage);
38. QTFile exportFile = new QTFile("StillImage.jpg");
39. graphicsExporter.setOutputFile(exportFile);
40. System.out.println("graphicsExporter: " + graphicsExporter);
42. // Do the export
43. int size = graphicsExporter.doExport();
44. byte[] jpegBytes = graphicsExporter.GraphicsExportReadOutputData(0, size);
45. System.out.println("jpegBytes: " + jpegBytes);
47. QTSession.close();
48. } catch (Exception e) {
49. QTSession.close();
50. System.out.println(e);
51. System.exit(0);
52. }
53. }
55. public static void main(String args[]) {
56. if (args.length < 2) {
57. System.out.println("You need to specify a URL and a location time in seconds when launching");
58. return;
59. }
60. GetImage instance = new GetImage();
61. instance.getImageFromMovie(args[0], args[1]);
62. }
63. }

Using the code

The following instructions assume you are using a Macintosh but are almost identical for other platforms. First, ensure that you have QuickTime for Java installed. Copy the code above into a text file and save the file as GetImage.java on the Desktop, or alternatively download the class in a .java file. Next, assuming you have a JDK installed, compile the source. Installing Apple's Xcode Tools will give you a Java compiler. Open the Terminal application and compile by typing:

javac -classpath /System/Library/Java/Extensions/QTJava.zip GetImage.java

A file named GetImage.class will be produced. Alternatively download the compiled application. To run the application, navigate to the directory containing the GetImage.class and type something like:

java -classpath /System/Library/Java/Extensions/QTJava.zip: GetImage http://localhost/Cremaster.mov 22

The application will save the result as a file named StillImage.jpg in the same directory. The image produced is not scaled but this is fairly easy to add in. The code sample above contains lots of debug logging so remove the System.out.println lines.

Tags: QuickTime, Java

For the last eight years Watershed has always distributed video over the Web using QuickTime .mov files. The last time we explored encoding and distribution standards was during the research stage of the Digitised project in 2001. As we start to think about dShed and the implications of a digital archive it's time to look at how we produce, and our users consume, video.

As all of the video content becomes part of the dShed archive we need to think about what is best for the future users of the archive not just the users of dShed this week.

The move away from Sorenson Video codec in QuickTime could be a painful process. Using MPEG4 pt10 files in Electric December 2005 meant that almost all of Watershed's staff were unable to view the video content. The public viewing computers around the building all required upgrades.

Apple have set the hardware and OS bar quite high for QuickTime 7, especially (and ironically) for Macintosh users. Many Windows users are reticent about upgrading to QuickTime 7 because the most prominently displayed installer installs iTunes as an 'added extra'. iTunes will probably become Apple's default media player in the future replicating the UI mistakes Real made almost 10 years ago.

The transition to MPEG4 will be a gradual process. Currently some QuickTime files use h264 and AAC encoding. Even when the files become fully compliant we will still push the files through the QuickTime plug-in to maintain a consistently good user experience (unless they don't have QuickTime installed).

Using h264 and AAC gives much smaller file sizes for the equivalent quality. This saving can be substantial with thousands of visitors viewing large video files. We get reduced bandwidth costs and the users spend less time downloading and more time viewing. Other advantages include local colour and gamma correction. Some people compress a different version with colour and gamma settings for each platform; we didn't. Previously, the gamma was set somewhere between that of a Macintosh and a Windows PC, with no user getting the best experience.

We may lose some content viewers by requiring QuickTime 7 to view the video in the browser but we will gain many more users by leveraging a common, standard media format that is supported by low power consumption, hardware decoders in a variety of portable devices. The iPod and PSP currently account for most of these, but their share will diminish as more and more mobile phones get the decoders.
New distribution methods, such as video podcasts, allow us to broadcast further to an audience that has 'opted in' to view specific content. After we added a 90 Second Challenge video podcast to the iTunes Music Store, even before video iPods were released, the feed accounted for 62% of all traffic to dShed.
In the short-term we may loose up to 30% of current potential viewers but will gain more than a 300% increase through new deployment opportunities.

The podcast distribution format appears almost too good to be true. It took half a day to add the podcast to the web application. The podcast is marketed by Apple (using iTMS) and other directory sites to a very large number of users at no cost or effort on our part. It is quite wondrous when comparing the time and money spent developing large content umbrella and directory sites. There may even be a situation where we don't need a web-site - just a podcast.

Tags: QuickTime