JVC Everio HD editing on a Mac: Pt II

This is a follow-up to my previous article in which I listed a variety of issues I've experienced in trying to edit JVC .TOD files on a Mac.

In this article, I will list some of my discoveries so far. If you're experiencing similar problems, you'll hopefully find this useful. If you've already found solutions then please let me know in the comments.

Summary of the issues:

1. JVCs MPEG2 .TOD files do not work natively in Mac OS X (10.4.11)
2. iMovie HD 6.0.3 will not properly convert the TOD files using the supplied Quicktime/Everio component - some video and audio is clipped from converted files leading to "jitter" glitches where several clips are combined in a finished video.

What I HAVE discovered so far:

Let me clear one thing up right at the start - renaming your .TOD files to .MPG files will not solve the problems. This reportedly used to work when JVC used .MOD files but it doesn't work with .TOD files.

The Everio camcorder imposes a 4Gb file limit - which equates to about 19 minutes of Hi-Definition video. This doesn't effect normal playback through the camera but can be a problem if you use iMovie HD to edit the files because iMovie (using the Quicktime/Everio Component I assume) drops some frames from the individual video clips.

In an effort to work out what was going on, I dragged one of the original .TOD files into ffmpegx, a Mac utility for converting all manner of video and audio files to all manner of different video and audio files (using the open source ffmpeg tools). I used this to convert the .TOD file to Quicktime (MOV) format. Unlike the iMovie-converted files, these converted files were not clipped.

What I discovered from this was that each file started with immediate audio but ten frames of still video. Each file ended with a few frames of silence (no audio) but video motion continued. What appears to happen here is that the Everio stops recording the audio shortly before it stops recording video. It then seamlessly generates the next video file (assuming you've recorded past the 19 minute limit) and starts recording audio to this new file before it starts recording video.

(My assumption about what happens with the video files)

From this it appears that the files are intended to overlap slightly for seamless playback. Presumably, the camera ignores the still frames in the next clip whilst it plays the new audio over the silence in clip 1.

I assume it works something like this...

It also appears that, for whatever reason, iMovie is trimming off the start of each clip so that in the iMovie editor, we don't see those ten frames (almost half a second in PAL format) of frozen video - but we also lose ten equivalent frames of audio as a result. We can easily lose a whole word if someone is speaking at this point in the clip.

When you put two consecutive clips together in iMovie, it might be something like this - with a split second of missing audio (which can be very noticeable, almost like a "pop"):

With this in mind I began to experiment.

I dragged a .TOD file into ffmpegx and exported an audio-only file from it. I tried AC3 format first but discovered iMovie doesn't recognise it (damn). So, I exported the audio as MP3. This takes a few minutes with a 3.42Gb .TOD file.

In an iMovie project that already contained some of the problematic video clips, I switched to Timeline view and dragged the new audio file into one of the audio tracks. Then I had to align it with the audio in the existing video clip. You can do this by making sure only the audio clip is selected then hitting the "PLAY" button. Make sure audio for both tracks is turned "ON" and listen for echo. If the gap is large, stop the video and drag the audio file closer into sync. If there is a slight echo then you can hit the left/right arrows on your keyboard to move the clip one frame either way until there's no audible echo. iMovie will let you do this while the clip continually replays from the start with each "nudge". (Make sure only one clip is selected or you could find the clip refuses to move or you could move other clips out of sync). Use the timeline "zoom" slider to make life a little easier (I constantly zoomed in and out whilst messing with this).

Be sure to check that the audio stays in sync for the whole clip. I found it drifted a tiny, tiny bit (less than a frame) but not enough that anyone would notice with only one sound track playing. From here you can disable the sound in the video file (turn off the audio tick box (check box)).

What you should find over a range of clips (and assuming the video hasn't been edited) is that the audio clips will start about ten frames earlier than the video clips. See screenshots below:


Note: I'm still working this stuff out and I'm writing it here firstly as a reminder to myself, secondly in the hope someone might offer better solutions and thirdly because it might help others solve their own problems. My suggestions and conclusions might change as I get more comfortable with messing with these files.

MORE TO COME

JVC Everio HD editing on a Mac

The following article represents my personal experience. If you feel I have got something completely wrong - TELL ME!

I purchased a JVC Everio GZ-HD30 Hi-definition (HD), Hard Disc Drive (HDD) camcorder late last year and after hours (and hours and hours) of finding ways to actually edit the video files on a Mac (OS X - 10.4.11), I've decided to log my trials, failures and successes here - mainly so I can refer back to them next time I hit a brick wall.

If you're having problems with a JVC Everio camcorder and your Mac, you can read this article to see if your problem is listed or you can jump to the follow-up article in which I list some of the things I've tried and what's worked so far (JVC Everio Editing on a Mac: Pt II). If you have any other solutions (or problems I guess), please leave a comment so I can update the articles.

First some details:

• The camera records only in 1080i Hi-Definition ("Full HD" = 1920 x 1080 pixels)
• The files are stored in a proprietary (JVC) ".TOD" format which is a type of MPEG2 stream*
  (*It will also record in AVCHD but I'm not using that for various reasons)
  
• The camera comes only with a USB 2.0 data connector (it has a variety of connectors for video output)
• The box includes a CD with a "QuickTime Component for Everio.pkg" (to make the TOD files readable in some applications)
• The box states the camera is compatible with a Mac with 1.25GHz processor & OS 10.4.4 - or better
• I can find no recommendations on the minimal software requirement for editing (but iMovie HD 6 seems to be the popular recommendation on various video forums)
• JVC do not provide any editing software for Mac users
  

Let me say up-front that to me, the quality of the HD footage is exceptional for a $1400 (Australian) camera. (I have nothing to compare it to since this is my first camcorder - but still, it is clear and sharp and rich.)

Let me also say that, for now, I don't really need the HD quality since DVDs are only Standard Definition (SD) but I purchased this with an eye on the near future where Hi-Def will likely be the standard. Plus, since editing requires re-encoding and burning to DVD requires re-encoding again, the extra definition means the end result is most probably better than if I began with SD video (every time a video is re-encoded, some quality is usually lost).

So let's begin...

Here is my simple plan:

1. Get the video footage from the camera into iMovie HD (6.0.3)
2. Edit the footage with transitions, titles and chapter markers
3. "Import" the iMovie project into iDVD
4. Add menus, etc
5. Burn a DVD (or Disc Image)

Not too much to ask for, you would think, from a camcorder made by a major consumer-products producer (JVC) and a computer (Apple Mac) designed for doing things like editing video and making quality DVDs. But you'd think wrong.

It appears that either JVC didn't think anyone used Macs or Apple didn't know JVC existed - or neither one particularly cared? Either way, the process of editing JVC .TOD files on a Mac (G5 PPC using Tiger 10.4.11) is anything but straightforward.

The issues:
iMovie HD 6 (on a PPC Mac) will only import directly from cameras using Firewire, not USB, so the camera must be mounted as an extral drive and the files copied in the Finder to a folder on the computer's drive (or external drive). This is a minor issue I can live with and it likely offers some added flexibility anyway.

It appears that Apple computers don't work natively with MPEG2 files - but you can purchase Apple's MPEG2-Playback Component. This makes the Mac "out of the box" - less-than-friendly for simple video/DVD production if camcorder producers use MPEG2 as their chosen format. JVC circumvent this requirement by including the QuickTime Component for Everio with the camera but it only allows limited access to the .TOD files. They cannot, for example, be opened by MPEG Streamclip - a popular (and FREE!) conversion tool. MPEG Streamclip will still complain that you need the Apple MPEG2-Playback component and will refuse to work with the .TOD files if you don't have it (despite what you might read elsewhere).

Although the supplied Quicktime/Everio Component does indeed allow the files to play in Quicktime and be imported into iMovie HD, the import process is very, very slow because iMovie has to convert the files to Quicktime format (using the Apple Intermediate Codec). On my dual 2GHz, 2GB RAM G5, each 4Gb file (actually 3.42Gb on the external drive) takes 70 minutes to convert!

But the conversion is imperfect! Occasionally you will notice that the first few frames in a clip are missing. This can result in a noticeable "pop" where there should be continuous audio. For most users, this problem will not be noticed. But there are times where the problem is very noticeable and attempts to fix it in iMovie by editing clips will lead to frustration and, usually, jumps in the video too. To understand this, we need a little background on Everio video files...

The Everio camcorder imposes a 4Gb file limit on video clips. This is around 19 minutes of video but it's okay because the camera seamlessly creates a new file on-the-fly if you record past this limit - so you can just keep filming until the camera's hard disc is full - or the battery runs out. If you playback your footage through the camera, you will not see the joins. This is fantastic! Well, it's great unless you then convert these files in iMovie HD using the supplied Quicktime/Everio component - then it's not so fantastic because those missing frames I mentioned above are needed for a seamless transition between clips. At this point you'll also find there's a bit of audio missing too so if someone was speaking when the new clip was created, you can miss all or part of a word (and see a noticeable jump in the video) - every 19 minutes at least.

A Google search will find lots of suggestions for a whole variety of ways to get around various problems with Everio .TOD files. Many people recommend MPEG Streamclip but, as already noted, it will not work with these files unless you hand over some more money to Apple and buy the MPEG2 -Playback component. (Personally, I wish JVC had just supplied a licensed copy of this component with the camera - it might well have solved a host of problems, but then again, it might not.)

So what does a Mac user do if they went ahead and bought an HD Everio assuming it would work on their computer? Can it be done without expensive (or even cheap) additonal software? After all, iMovie is the Mac tool of choice for this task for most home users and it is no longer free. I'll go through some of my findings in the next article: JVC Everio Editing on a Mac: Pt II.

Digital TV on Sony DVD Recorder

I'm thinking of making this blog in more generic since I don't get much time to look at Blender these days.

Can't get digital TV stations on DVD Recorder:

I recently purchased a Sony HXD890 HDD DVD Recorder and struggled to get it to display the digital stations I knew were available in my region, ABC2 and SBS Digital (There are HD stations too but it's only an SD tuner). All analogue stations displayed as expected.

The manual says to press the "input button" to switch from Analogue to Digital tuner mode. I did this but pressing "input" just scrolled through Line 1 (L1), Line 2 (L2) and the Analogue program. No digital. Even going to channel 11 or 29 didn't work.

I went into the setup menu and checked the signal strength. Both channels (11 & 29) came up at 100%. I noticed only some channel numbers were available and assumed this meant they were preset, where available. I was wrong.

Though not made clear in the manual or when navigating the the menu (not clear to me anyway), it was necessary to "replace" the channels - that it seems aren't actually there in the first place. The first hint of this should have been that some channel options were greyed out (unavailable) but as a newcomer to digital, this meant nothing to me - and living in regional WA, I'm used to lots of things being unavailable.

"Replace Channels" meant nothing to me since I hadn't programmed any channels and had no intention of messing up the presets I assumed would be there. I was thrown because I was able to check the signal strength and had assumed that the unit would therefore recognise channels 11 and 29 as digital when changing programs but it doesn't work like that.

So, in short, if you're suffering the same dilemma, go to the setup menu, choose the Digital Tuner option and select "REPLACE CHANNELS" to have the unit scan for all available digital frequencies. I got 13 channels, including some "duplicates" and radio stations.

Headache over.

Game Engine Vehicle Physics

I've recently turned my attention to the Blender Game Engine and have to say I'm incredibly impressed by the possibilities. However, as with any new venture, it comes a the price of having to learn some things from scratch - like physics.

I'll have more to say about the game engine in later posts but this one is to hopefully get a problem solved.

I've been messing with the vehicle physics simulations and having an absolute ball until today. I've discovered that if I use a vehicle to knock over an obstacle, I cannot then drive over that obstacle like any normal object. As soon as the vehicle attempts to go over the toppled obstacle, it lurches and jumps and launches away at speed. The obstacle is usually pushed violently through the floor even though the initial contact was at low speed.

Here's a rough OGL video capture of a simulation. The first obstacle is a flattened cube and the car goes over normally. The second obstacle is another flattened cube but because it falls over, it has an undesirable effect on the car. Both obstacles are rigid bodies.



Note to non-Blender users, new Blender users and potential Blender users: The above video is NOT intended to showcase Blender. It's just a rough demonstration of a problem.

Missing DVD Features - Error or Scam?

No, this isn't about Disney-Pixar's decision to release single-disc DVDs with very few features. This is about an entirely different problem - DVD case covers that promise a lot whilst containing DVDs that deliver very little.

In January this year I purchased A Few Good Men, 10th Anniversary Collector's Edition. The package details claim the disc contains a range of special features, including audio commentary, a 35 minute documentary, a 14 minute featurette and more. In fact, the disc contained none of these things.

The Region 4 DVD Glitch List notes that this is a "wrong disc" error and provides a phone number for a replacement disc. My wife called this number and was advised a replacement would be sent as soon as possible. It's now almost June and we've received not so much as a letter or a phone call - and definitely no replacement DVD.

This week we bought Men in Black, Deluxe Collector's Edition. The front cover boasts "Great special features. Over 9 hours of fun". The back cover lists a range of special effects features, a behind the scenes featurette and a lot more. No mention is made on the cover of this being a two-disc set and indeed, only one disc is included. However, when the features are selected from the menu, the viewer is advised to insert Disc 2. That's difficult when there isn't one.

Both discs are from Columbia Pictures.

Is this just bad luck or is it a major problem plaguing the industry?

Shawn Kelly Tips & Tricks - FREE!

I'll never cease to be amazed by the numbers of professionals in the animation industry who willingly pass on their valuable experience for free. Now we can add Shawn Kelly to the list.

Shawn is a founding member of AnimationMentor.com and has a solid list of animation credits under his belt. He's been writing his tips and tricks for members of AnimationMentor.com for a few years and since anyone can subscribe to the AnimationMentor newsletter for free, I guess we can say Shawn's tips have always been free. But now he's gone one better, compiling all past tips into a 99 page eBook!

Topics covered in this "treasure trove of information" on character animation include Planning, Observation, Blinking, The Face, Expressions, Exaggeration, Walks, Arcs, Timing and, well, lots more. In short, if you're learning animation then you can't go wrong to have this eBook in your eLibrary.

Get the eBook HERE and while you're at it, you might as well sign up for the AnimationMentor Newsletter too so you get more of Shawn's tips as he writes them.

Wacom Woes

WHEN GOOD IDEAS GO BAD

UPDATE: Problem solved!

I purchased a Wacom Graphire 4 tablet last year and within a couple of weeks of using it, I filed my mouse in the bottom drawer. Driving graphics programs, especially painting applications, with a pen just makes sense but I was surprised how the pen also makes even mundane tasks like menu selection and shifting files around easier. It's just more comfortable than any mouse I've ever used - and, unlike a lot of mice with more than one button, the pen doesn't care that I'm left-handed.

BUT - and it's a big but (and, unlike Sir Mixalot, I don't particularly like "big buts") - the pen has one major failing. The Graphire 4 pen comes with a comfy rubber grip that supposedly makes it superior to older Wacom pens - but it is too flimsy to do the job.

At first I thought it was just a matter of me getting used to holding the pen in such a way as to make the buttons accessible but, as time went by, it became apparent the rubber grip was too loose to stay put. Eventually it has become so loose that it actually twists around the body and on more than one occasion it's covered the buttons, holding them depressed (I know how they feel). This week I've also noticed that the very thin slip of rubber between the buttons is starting to split.

EDITED

UPDATE

After a few emails to Wacom, I received a replacement pen from them this week. The new pen is a different model with a single toggle switch and no rubber grip. After one night, I already love this pen whereas I never was comfortable with the grip pen. Thanks Wacom.