This post is brought to you by the letters ‘K’ & ‘J’

Another nice application of a slitscan process, this time using fonts can be seen at :http://www.c71123.com/typography/slitscan-type-project/

This approach uses Adobe Illustrator CS or CS2, and a script provided by the project’s author. Tré cool.

The outcome of all this testing and tweaking

I produced another image using the video slicing technique. The input footage was a slowly panning shot taken at the National Museum of Australia. I have more of these to process, but the files are not available to me at the moment. I think some of the other footage will produce better images.

Workflow for slice and dice

OK, I gotta say it: “iMovie is crap”. Basically all the problems I had with video transport disappeared when I converted the DV footage to a suitable Quicktime supported codec using MPEG Streamclip.

So the workflow on Mac OS X is like this:

• Obtain footage in DV format or from another source (MPEG, AVI or whatever). , and iMovie is OK for getting the DV footage off a camera, but don’t use the ’share’ feature to convert it to some other format.
• Grab the DV footage if you have to. If it was downloaded to the computer by iMovie, find the movie project, select it and use the ‘Show package contents’ of OS X (see the ‘cog’ dropdown menu on the enclosing folder’s window or control-click and use the contextual menu item of the same name). You will then see a folder that contains the .dv files.
• Convert the DV file (or whatever format you have), using MPEG Streamclip. It’s available from www.squared5.com/ and various other download sites on the internet. I found the Apple Motion JPEG A format that presumably is part of the Quicktime installation to be reasonable compromise for speed, file size and quality. The A and B formats are apparently a legacy from the codec’s earlier days, and now it shouldn’t matter which of the two are used. At 85% quality which is supposed to be ‘visually lossless’ the file should compress to around 60% of the DV size, but I guess the results may vary depending on the source material.
• So after this you should have a .mov file that can be further processed in processing. For my project I use a slicing technique that is basically a slitscan, and it can be run interactively or to pre-recorded points in order to manipulate where the pixe-wide slices of video are taken from.

The problems I was having with iMovie were basically to do with the sync points or ‘interesting time’ markers that it was inserting into converted files. It was putting extra points in which made no sense in the context of where the individual frames began/ended. Also the H.264 format didn’t work correctly with some quicktime.Media object methods, yet I have since tested a H.264 format movie exported by MPEG Streamclip that did not seem to suffer from the same problems so I can only assume that iMovie mings this up too.

I tested a number of different codecs at higher quality settings with a DV file that was around 160 MB. Platform was my ancient 450MHz Blue and White G3 Mac running OS X 10.3.9 and Quicktime 7.1.6. The original file specs:

• DV – PAL, 720 × 576, 4:3, 25 fps, 28.80 Mbps
  Audio Tracks: DV stereo, 48 kHz, 28.80 Mbps
  Frame rate in Quicktime Player ~8 fps actual.

The exported files had no sound exported with them as it was not required. I would expect the frame rate improved (in some cases) because of this. Performance in Quicktime Player was as follows:

• Apple DV PAL
  Frame rate in Quicktime Player ~18 fps actual.
• Apple Motion JPEG A
  Frame rate in Quicktime Player ~15 fps actual
• MPEG4
  Unusably slow frame rate in Quicktime Player
• Sorenson 3
  Unusably slow frame rate in Quicktime Player
• 3ivxD4
  Unusably slow frame rate in Quicktime Player

A bit of math…

I’m using one of the Panasonic DV cameras (model ag-dvc30e) to record some video. Seeing as I would like to come up with a rough guide to camera panning (and perhaps movement through space), I needed to figure some stuff out.

The Panasonic’s digital multiplier is ~9.6. This figure was not quoted directly in any info I could find but was calculated from the 35mm equivalent figures.

The reciprocal of 9.6 is:
1/9.6 = 0.1041667

Which gives a CCD element size of:
0.1041667 * 36mm = 3.75mm
0.1041667 * 24mm = 2.5mm
The 36mm and 24mm above are 35mm film dimensions.

FOV (rectilinear) = 2 * arctan(frame size/(focal length * 2))
(source http://photo.net/learn/fov/)

So by substitution we have for the minimum zoom setting:
HFOV = 2 * arctan(3.75/(4.1 * 2)) ~= 49.15 degrees
VFOV = 2 * arctan(2.5/(4.1 * 2)) ~= 33.91 degrees

And for the maximum zoom setting:
HFOV = 2 * arctan(3.75/(65.6 * 2)) ~= 3.27 degrees
VFOV = 2 * arctan(2.5/(65.6 * 2)) ~= 1.09 degrees
The focal length figures for 4.1 and 65.6mm are taken from the tech spec sheet for the camera found online somewhere.

These figures actually seem accurate despite the guesswork and approximations. The horizontal FOV matched up pretty well when I checked by eyeballing the FOV on the camera’s viewscreen against lines of sight taken from a protractor.

784 pixels horizontally per frame.
~16pixels per degree.
~5760 pixels for 360 degrees.
So 5760 pixels divided by 25 fps gives ~230 seconds for a 360 degree horizontal pan moving at a rate of one pixel per frame.

The original pan I took was around 36 seconds in duration, and 230 / 36 ~= 6.39 pixels per frame horizontal movement. I’m pretty sure this matches up closely to the average number of slices taken when tracking the horizontal movement of that clip (found when I was doing some variable width slice experiments), so the figure of 230 seconds must be a pretty good guide for how long a 360 degree pan should take.

That’s kind of a long time, and a small movement rate to pan at manually, but it’s not too bad with a tripod, practice and planning. Motion control would be nice too though.

Out of memergy error

OK, the Java Virtual Machine doesn’t like it when you try to use more memory than processing has allocated.

A compressed movie may be a few MB or less, but when it is made in to a video cube the size can grow massively. For example a 2.4MB MotionJPEG movie takes up nearly 160MB as a serialized object. Once the size of the videocube object exceeds the memory allocation things break and out of memory errors ensue.

The lab G5’s have 256MB of memory and processing can be allocated all of that, but it’s not really going to cut it with even a 5MB movie. Presumably giving processing all the memory will result in page swapping because the OS needs something too, so performance will not be so great either.

I guess this is just a limitation of the videocube idea. Not sure how to deal with it.

Assesment ideas

Well time is drawing near ot wrap this stuff up. This post is just a quick draft of what I’ll try to get together for assessment.

• The PMovie library for processing – will need to get it in order by adding a few more methods and cleaning things up, putting a bit more documentation together for it.
• The other classes I wrote for processing may need a bit of attention. These Classes should form part of my assessment: VideoCube, VideoTracking, and Serialize (maybe – this is just a general utility really).
• The videocube work that runs vectors through the cube. More variations and finished stills/movies/sketches required for this one.
• Video slicing (or slitscan) works. Will need to shoot more video for these to obtain suitable footage with steady and very slow camera movement. Having tried taking slices greater than a few pixels wide, I think I will settle with the 1 or 2 pixel wide slices that I started out my experiments with. Again the assessment should include number of stills/movies/sketches of high quality.
• The work with supershapes hasn’t really progressed much from a basic implementation. If time allows I would like to try a recursive application, and maybe a hybrid use of it with a videocube or slicing techniques.
• I will produce a ‘written’ synopsis of the video techniques I have encountered in the course of my research this semester. Basically I will either write a series of blogs or a series of webpages describing some works and approaches. I’m favouring the latter idea of webpages at the moment because it is a little easier to include samples, applets, links etc. and the control over the design and format is much better.
• Very much doubt that a 360 degree video application will be done at all. I did some research, but have not made a rig, shot footage, or made anything to de-warp the footage to a rectilinear or cylindrical projection.

Hopefully this will be enough material. Not much of it is at present ready for submission, and I need to inform Mitchell of my submission date. Probably late in the second week of exams if that’s not pushing it.