The Jellyfish and Final Cut Pro X go on the road for a real broadcast job. Cutting edge portable shared storage on a budget, but with all of the performance you need. How did it cope with the strain of continuous records and multiple edit clients? Peter Wiggins takes the new cnidarian for a mission critical swim.
I get asked to edit on many remote and outside broadcast jobs. Sometimes you get to choose what equipment you want to work with and there are others where you get the producers', production managers' or engineers' choice.
The recent World Track Cycling Championship in London was neither. During a telephone conversation between myself and the producer, James from VSquared, we came to the conclusion that we probably had enough Macs and ancillary kit between us to do the job on FCPX.
This idea was triggered by the Scottish Labour Party Conference I edited for Tern TV for BBC Scotland last year. Fellow editor John Matthews from Blue Bicycle had put together a collection of Macs that all working together; did a superb job of making a programme out of two live feeds and camera originals. Storage was Thunderbolt RAIDs shared out via ethernet from three machines. Not ideal for computer purists, but it worked a treat. I was impressed.
Cutting Labour's Jeremy Corbyn down to size on FCPX last year - He's not chromakeyed on a media missing warning!
Rather than 3 days, this would be 5 with an extra edit day at the end. We would also have to have another edit suite working all day off growing files. They would also have to have access to the camera originals which were being shot on an Sony F5 and ingested via SxS.
What was missing was proper shared storage. This is normally expensive, bulky and difficult to administer.
So by now I'm pretty sure you've realised the reason Sam Mestman from LumaForge lent me the first Jellyfish in the country, if not Europe. It was to be the centralised shared storage system for this event.
There were multiple requirements during the job. The first was to edit up any packages that the director required. This might be the top of the show or highlights from previous days or rounds. There was to be a 52 minute voiced highlights show of the whole event to be delivered and each day 10-30 clips were required to be quickly uploaded to social media platforms.
I handled the first two, a second editor Sylvain Swimer, wrangled the media off to YouTube and Facebook.
The edit kit comprised of the Jellyfish, a 27inch Retina iMac connected via 10GigE, a 27 inch iMac connected via GigE and two laptops, again both connected with GigE.
The two iMacs were for editing, one of the MBPs ran the excellent MovieRecorder from Softron to record the TX output and the other did an incremental backup to a Pegasus RAID using Chronosync. Using the incremental backups of Chronosync would mean that by power-off each day we would have a separate copy of the media in addition to the Jellyfish.
The first thing to do after the Jellyfish arrived was a soak test. For five days I booted up the system, set MovieRecorder going in 1 hour chunk records, loaded a growing clip into FCPX on both iMacs to play and then left the whole setup running for twelve hours. I powered off everything at night as I knew we would be working off a generator on location.
The Blackmagic Hyperdeck was borrowed to act as a continuous HD-SDI source. That fed a Blackmagic Ultrastudio Express via Thunderbolt to the MacBook Pro.
Whilst the test was 'cooking' I had the opportunity to make some opening titles. Again the 3D capabilities of Motion projects published to FCPX let me build some very tasty looking world championship stripes. All done in half a day.
I seem to be using Motion's & FCPX's 3D capabilities more and more on broadcast jobs. Turning up with a 'palette' of custom effects helps speed things up and puts a lot of gloss on the output.
Building the animating World Champion rings in Motion to be published to FCPX.
So five days of continuous record on, not one frame dropped by MovieRecorder and I was happy. So everything was packed up ready to go to the Velodrome on the old Olympic Park in London.
The Jellyfish set up at the back of the main area in the OB truck. It is pretty quiet so it went on a desk right behind the main director! On top is the MBP running MovieRecorder and to the left the other MBP (screen dimmed) running Chronosync.
My edit position was on the other side of a glass door at the back. Not the most spacious of places to work in, but I've had a lot worse. The 27" Retina iMac's small footprint was ideal for the desk allowing space for the AP on the left.
GRAID Thunderbolt drives on the left only got connected when archive was needed.
The iMac is a truly great edit machine with a screen to die for. You can read why I like it so much here:
One problem though is that there are only two Thunderbolt ports on the back. This probably won't bother you if you are using GigE for storage, but using 10GigE and a Thunderbolt to Cat7 box means that you will run out of ports if you have a broadcast monitor connected with an I/O box.
Only two Thunderbolt ports on the back of the iMac.
During the 5 transmission days, I ingested 1.25 TB of slomo 1080p media from the F5. A pain as I had to do a cable swap with the Blackmagic Ultrastudio Express and a reboot sometimes. The iMac failed to recognise the SxS reader as it still thought it was in video out mode. There is another port on the 10GigE box that would have worked as a loop through or a third party expansion box could have been used. I'll remember that for next time!
Swapping Thunderbolt cables between peripherals.
To this date, Blackmagic has still not fixed the problem of crashes using the broadcast output on El Capitan. Not a problem as I ended up using the internal speakers for most of the days as the main audio monitoring was high up in the rack above my head. I was quite surprised how good it was!
On the next desk in front of me was Sylvain Swimer who was servicing all the needs of social media on his iMac. He had access to all of the growing files and camera originals from the F5. He also had a very fast internet link which was important as the minimum time delay from event to publishing is critical for getting good viewing stats.
Sylvain on the left and VT Supervisor Chris Fells from Observe on the right. Sylv isn't going bald, it was an LED spotlight pointed at his head!
On the first day everything went amazingly well. For a system put together from scrounged and borrowed items it was literally cutting edge.
Day two and things didn't go so well. The Jellyfish would not boot. Sylvain was straight in to have a look as he had been itching to get the lid of the moment he had seen the box.
We traced the problem down to the boot drive which had a fault. An early indication of this had been that the HTML GUI of the Jellyfish had been slowing down slightly over time. The unit that Sam had supplied was a prototype, not a production model and although it had survived the transatlantic journey, it wasn't too happy now. Some of the components had become loose and caused the issue.
Sam and Eric from Lumaforge talked Sylvain through making another boot drive and after a bit of remote tinkering to reconfigure the machine all the way from California, we were back in business. We did not lose any of the footage on the Jellyfish.
We always knew we were on the edge by using a prototype, so we had a couple of backup plans in action. Whilst the Jellyfish was being repaired, we gave the EVS streams on the truck a go in FCPX.
They say every cloud has a silver lining and this was the first time I had ever seen a growing EVS stream working with the correct timecode in FCPX. The reference movies were no where near as responsive as the MovieRecorder QuickTimes and the recordings were running at least 5 minutes behind the action. With MovieRecorder you can get to within a few seconds of real time. But they were there.
Sam and Eric let us know that we had hit their worst case scenario and had hit an oversight on the prototype model that wasn’t going to be present in actual shipping models of the Jellyfish. The fact is that hard drives fail on occasion, but with shipping models of the Jellyfish, Boot drives are going to be mirrored and built right onto the system (as well as being placed in a custom box designed for overseas shipping), meaning that even in the event of a boot drive failure, you will never be at risk of prolonged down time with a shipping unit. Should your boot drive fail, your data is not tied to it, so none of your work will be lost, which is different from how most traditional shared storage units operate.
One other slight hiccup we had was with a recording stopping. Not a fault with the bullet proof MovieRecorder, but the MacBook Pro that it was being run on. Somehow, the MagSafe mains connector had become dislodged and although it carried on recording using the battery for a while, eventually the MBP shut down.
If you do use laptops for unattended jobs such as feed recording, make sure the mains connection cannot be easily pulled out. Another one to chalk up to experience!
Make sure MagSafe connectors cannot be easily pulled out!
We had another power problem, although we knew this from the start. On the last day, the racing would finish about 1800, but we would only have truck power until 2000. Which was certainly not enough time to finish off the 52 minute highlight show.
Instead, we concentrated on making sure we had all the media on the Jellyfish and that Chronosync had made a complete copy on to the Pegasus. One interesting find here. Chronosync will report an error and not copy a growing file, but once that file is closed, on the subsequent backup cycle it will be copied without a problem.
As we had set backups to run at 15 minutes past and 15 minutes to the hour, we were only ever half an hour or so away from having a complete copy. Kudos again to Chris Roberts for showing me how valuable a $49 piece of software can be in reducing the headache of copying drives.
The day after the event, a room had been booked inside the Velodrome and all the kit got set up on tables. I could finesse the audio mix now that we were out of the truck, the AP could look through the F5 footage (on the other iMac) for great shots we might have missed and the producer could copy off various segments of racing that were needed for the organisers. All from the Jellyfish and all of us working at once.
The final 52 minute highlight programme. (Open for larger image)
Then it struck me. This was truly portable shared storage. Three people working on a system that 18 hours ago had been running in an OB truck. No copying required, no waiting for drives, it was just plug it up and finish the job. Which we did 10 minutes before the Velodrome staff told us they were shutting the venue at 2100.
By then I had exported a master out of FCPX using roles so that we could deliver a full English comms mix on audios 1&2 and international sound or M&E on 3&4. After a drive back to the office (The fast internet was not available) I uploaded the programme to Frame.io.
This meant that that AP could watch the show at leisure at his home in the morning. Good job he did too as we had inadvertently got the wrong shot of a rider in- a bit tricky when they are all wearing the same colours on the same team and of course all wear helmets!
No problem, the Jellyfish and iMac was powered up, the shot changed and then the ProRes 422 master uploaded for distribution. Easy and fast.
The next day we just had to catch up on a bit of paperwork - the dreaded music returns! Not a problem as an FCPXML of the sequence in Producer's Best Friend allows you to export an Excel spreadsheet of all the music used. With timings of course.
One bug, it didn't report the music that was assigned with the music role on an audio component - this was obvious as it didn't report the music on the opening titles. A few email exchanges later with Greg Clarke from Intelligent Assistance and I'm pleased to report that the updated version on the Mac App Store now works perfectly. Can't beat that for customer service!
So my conclusion? Pretty amazing, flexible, fast and inexpensive, definitely the way forward to cover live events. Desktop shared storage is here. Yes we had the hiccup with the Jellyfish, but once it was fixed I can only give it the best compliment I can: I forgot it was there and got on with the editing. It has since been in car boots, done many miles and not had one problem.
Thank you to everybody for their help. James & Carolyn, Sam & Eric and Nic, Chris & Sylvain
Peter Wiggins is a broadcast freelance editor based in the UK although his work takes him around the world. An early adopter of FCP setting up pioneering broadcasts workflows, his weapon of choice is now Final Cut Pro X. You can find him on Twitter as @peterwiggins or as he runs the majority of this site, you can contact him here.