Advertisement
Current Issue
December 2014

Recent Blog Posts in December 2012

December 17, 2012
  Don't Waste Your Money On Faster Storage
Posted By Larry Jordan
Most of the time when we work on our computers, the files we create are relatively small. We load the files into the RAM memory of our computer and work away.

Working with media, however, is different. Media files tend to be massive. Most often, they need to be played in real-time and they are too big to be fully stored in RAM memory.

Why am I saying "RAM memory?" Because many people use the term "memory" to mean "hard disk storage." For me, computer "memory" means RAM - that which disappears when power to the computer is turned off.

The "Data Transfer Rate" is the speed data travels between your hard disk storage and the computer, and it is measured in "bits per second"

To keep the numbers from getting too big when discussing hard disks, bps is often converted to MBps, or "megabytes per second." The conversion equation is simple: bps/8,000,000 = MBps. (Engineers sometimes use 8,388,608 instead of 8 million.) We also write MBps as MB/s.

There are three things that determine the speed of your data between the hard disk and your computer:

- How you connect your storage (USB 2, USB 3, FireWire, Thunderbolt...)
- The number of hard disks in your device (a hard drive has one disk, RAIDs have several)
- The internal hardware architecture of your storage device

CABLES AND PROTOCOLS

Changing how you connect your hard disk changes the speed of your data.

For example, USB 2 transfers data around 15 MB/s, FireWire 400 transfers data about 25 MB/s, while FireWire 800 is around 85 MB/s. If you spent money and bought an infinitely fast drive, then connected it to your computer via USB 2, your data would only travel at 15 MB/s, which is the speed limit of USB 2.

There's a lot of conversation today about the speed of SSD drives. SSD, which stands for Solid State Drive, can be very fast. But, if you attach an SSD drive using a very slow protocol - say, FireWire 400 - you'll never see the speed that SSD can deliver. The FireWire 400 protocol is too slow.

NUMBER OF HARD DRIVES

Connected internally, directly to the data bus of your computer, a single standard 3.5" hard drive delivers about 120 MB/s of data.

If we connect that drive via USB 2, FireWire 400, or FireWire 800, the connection protocol is slower than that of the drive, so we can't get the maximum speed from the drive because it is limited by the protocol.

However, other connection protocols, such as USB 3 and Thunderbolt can transfer data at much faster speeds than a single hard drive. USB 3 maxes out around 480 MB/s, while Thunderbolt can deliver up to 1.1 GB/s.

If you are attaching a single hard drive via USB 3, the fastest speed you can expect is about 120 MB/s, which is the maximum speed a single hard drive can deliver.

Here, the limitation is not the protocol, but the speed of the hard disk.

RAIDs allow us to combine multiple hard discs into a single unit. Now, we are able to combine the speeds from multiple hard drives so that the speed of the RAID is greater than the speed of a single hard drive.

At this point, the maximum speed of a RAID is the SLOWER of the sum of the speed of all the hard drives it contains, or the protocol that connects the RAID.

For example, a 2-drive RAID attached via USB 3 would transfer data at 240 MB/s (the sum of the two hard drives), while a 20-drive RAID attached via USB 3 would transfer data at 480 MB/s (the maximum transfer speed of USB 3).

SYSTEM OVERHEAD

Even when you've picked the fastest protocol, and connected a gajillion hard disks, there's one more factor that determines overall data transfer rate: system overhead.

This gets really complicated really quickly, however, the short answer is that every hard disk, RAID, and protocol needs to process the data both before it is sent and after it is received.  And this processing takes time.

Some manufacturers opt for the fastest data transfer speeds, but sacrifice flexibility. Others opt for greater flexibility, but sacrifice speed. Some do their data processing in hardware, others use software. Some opt for maximum speed, others for data security. It is almost impossible to figure out in advance how much performance will be lost due to system overhead.

For this reason, always be skeptical when a storage vendor describes their speeds using phrases like: "Up to 480 MB/s," or "Up to 1.1 GB/s."  They are quoting the speed of the protocol, not the speed of their device.

ACCURATE ESTIMATES

The easiest way to estimate the speed of a hard drive or RAID is to multiply the number of hard drives it contains by 100 MB/s.  This provides a ballpark range of the data speed to expect.

CONCLUSION

If all you are doing is editing a single stream of AVCHD video, any hard disk and protocol will be fine.

As you start to edit multiple streams of video, migrate to larger image sizes, or upgrade to more professional video formats, the speed of your storage system becomes critical.

Here are some simple rules:

- A single hard drive can deliver about 120 MB/s of data
- USB 2, FireWire 400, and FireWire 800 are slower than a single drive
- USB 3 and Thunderbolt are faster than a single drive
- If your computer supports plug-in cards, eSATA and mini-SAS are also very fast protocols
- RAIDs are always faster than single drives
- RAID speeds are, generally, the sum of the speeds of the drives they contain


Here's an article that can help you learn the differences between RAIDs, SSDs, and The Cloud.

Larry Jordan is a producer, director, author, editor, and Apple Certified Trainer with more than 35 years of professional experience. Based in Los Angeles, he is also a member of the Directors Guild of America and the Producers Guild of America, and author of eight books on Final Cut Pro and Adobe Premiere Pro. Visit his website at http://www.larryjordan.biz.
Continue reading "Don't Waste Your Money On Faster Storage" »

Permalink | Comments(0)
 
December 06, 2012
  Configuring an iMac for video editing
Posted By Larry Jordan
By Larry Jordan
larry@larryjordan.biz

This blog was first published on my Website, and is a good follow up to my last Post blog http://www.postmagazine.com/Post-Blog/2012/November/Are-the-new-iMacs-good-enough-for-video-editing-.aspx

I bought a new 27-inch iMac when they went on sale Friday, specifically for video editing. And, because I've had a lot of requests recently, I wanted to tell you what I bought and why.

WHAT SIZE IMAC?
I bought: 27-inch iMac. All versions of Final Cut Pro and Adobe Premiere like large screen sizes. It allows us to see more of the image with more detail. In my case, because this system is exclusively for video and audio editing, the bigger screen was an easy decision.

I also have a second 27-inch Apple monitor sitting unused on a shelf that I want to experiment with. I've generally found dual monitor displays at client sites to be more trouble than they are worth. But, I've never worked with one for a long period of time, so I'm looking forward to seeing how this new set-up works.

However, for my Webinars, I use a smaller 21-inch Mac, because I find software easier to learn when the screen sizes are kept smaller.

CPU SPEED
I bought: 3.4 GHz, Quad-Core Intel Core i7
CPU speed is important, but it isn't everything. The speed and connection of your storage play a much bigger role in overall system performance than the CPU. So does the speed of the graphics card.

In the old days, the CPU did all the work. Today, that load is shared between a variety of components. For this reason, I decided to get a fast CPU, but use the money I saved in not buying the fastest CPU to getting faster storage. Especially for multicam work, faster storage provides more benefits than a faster CPU.

Given the speed of today's processors, just about any CPU is more than fast enough to edit any flavor of HD video.

DRIVE
I bought: 1TB Fusion drive

This new technology from Apple combines the speed of SSD (Solid State Drive) with the storage capacity of standard spinning hard disks.
However, the Fusion drive delivers the fastest speeds when it is accessing the same material over and over. This means that it is optimized for the operating system and applications. Since we are constantly changing media, a Fusion drive won't deliver the same level of performance with our media.
I have long been a fan of storing media to a separate drive, rather than on the boot drive. In the past, this was primarily for performance reasons. Now, the internal drive is faster, but an external drive allows far more storage and flexibility.

I strongly recommend using an external RAID system, connected via USB 3, or Thunderbolt (more on that in a bit), because it will store more than any single internal drive, provide more than enough speed, protect your data using the data redundancy in the RAID, and allow easy upgrading by simply swapping out devices.

For me, the ideal situation is the Fusion drive for the OS, and an external RAID-5 for all media.

RAM
I bought: 16GB RAM

Both Premiere and FCP will use all the RAM you have available. So will video compression software. 16GB is a nice balance between performance and price. And, unless you are creating some truly massive edits, you won't notice enough difference between 16 and 32GB of RAM to justify the additional cost.

GRAPHICS CARD (GPU)
I bought: Nvidia GeForce GTX 675MX

This was a harder decision. Both Premiere CS6 and Final Cut X take advantage of the graphics card. However, in the CS6 release, Adobe only initially supported the graphics cards in the MacBook Pro. (Traditionally, Adobe only supports Nvidia cards and all the Apple gear uses ATI, which is now AMD.)

Now that the new iMacs include Nvidia, I'm hoping (but do NOT know for sure) that Adobe will quickly support the graphics cards in these new Macs. I've sent a note off to my friends at Adobe to see what I can learn and will let you know what I find out.

[NOTE: Even if Adobe doesn't support the graphics cards, Premiere Pro CS6 will run perfectly OK using just the CPU. It won't do as much, or work as fast as when the graphics card is involved, but you can still use Premiere on these new systems.
This isn't the fastest GPU that's available, but it is the second fastest. Again, for me, this was a balance between performance and price. Video editing requires a fast overall system, balanced amongst all the major components.]

GPU RAM
I bought: 1GB GDDRS

The RAM in a graphics card determines how many elements, for example frames of video, it can store for processing.

3D software and Motion makes extensive use of GPU RAM. However, video editors are using it principally for pixel painting. Since I am an editor more than a motion graphics designer, I don't need the extra GPU RAM. So, I stayed with the base level of 1GB.

KEYBOARD AND MOUSE
I bought: Apple keyboard with Numeric Keypad (wired) and mouse (wired)
Wireless gear is great, until your system starts acting up. At which point, you need a wired keyboard for maintenance. Also, there are a number of very useful keyboard shortcuts in all my applications that take advantage of the keypad.

If I were shooting a television commercial, I'd use a wireless keyboard and mouse because it looks cool on camera. Because I am editing television commercials, I'm using a wired keyboard and mouse because they work great, decrease my stress, allow me to easily do maintenance on my system, and don't require batteries.

EXTERNAL STORAGE
I bought: (um, nothing yet)

Since ProRes 422 is the default video codec of Final Cut Pro X, and a great codec to use for Premiere, I need storage that is big enough and fast enough to handle this format.

Prores 422 requires about 18MB/second of data transfer between the computer and storage. Because much of what I shoot is 3-5 camera multicam projects, this means I need to move about 100MB/second of data.

The problem is that FireWire 800 tops out around 80-85MB/second. Gigabit Ethernet tops out around 100MB/second, assuming your switch and server can handle the speeds, and most data switches that cost less than $200 can't handle that much data over a long period of time.

[NOTE: A "switch" is a device that allows multiple computers to connect to the Internet or a server by switching data from one device to another. These are made by NetGear, LinkSys, Cisco and others. A "server" is a computer with a large hard disk or RAID that allows multiple computers to share the same files. Servers can be a simple as a Mac Mini, or as complex as an Avid Isis system.]

This means that I need storage that connects via either USB 3, or Thunderbolt. (This is an iMac, which means that plug-in cards are not an option.) Yes, I could buy converter boxes - for example, from Thunderbolt to eSATA, or mini-SAS, but these boxes cost several hundred dollars apiece. If I were integrating existing hardware, this would be an inexpensive way to go. However, I'm buying all new gear.

It is at this point that I'm puzzled about why storage vendors are having such a hard time shipping RAID-5 Thunderbolt-based storage devices. Yes, Promise Technology is out there, and they recently dropped their prices, but where are the traditional storage vendors? G-Technology and LaCie both offer RAID-0 (which is fast, but provides no data safety in the event one of the hard drives in the unit dies), but no RAID-5. Drobo was way late in shipping their Thunderbolt storage, and I haven't had a chance to look at the shipping product. And, as far as I know, traditional RAID vendors haven't even announced RAID-5 storage with Thunderbolt connectivity. [Editor's note: G-Tech plans to ship their Thunderbolt RAID-5 in 2013. No specific date was offered.]

It is troubling to me that this new format is taking so l ong to take shape and appear in quantity in the market. Is this a licensing issue? Technical or integration issues? Are there hidden problems inherent with the Thunderbolt format that are holding things up? I have been inquiring about this for months and have not gotten a clear answer from any vendor.
So, I decided to hold off buying storage until I could do more research. My iMac is still a month away from shipping, so I have some time to figure this out.

ARCHIVING

I bought: (also, nothing yet)
Long-term data storage, today, means LTO tape. The problem is that all the tape vendors - Cache-A, The Tolis Group, Xendata - provide solutions much closer to $10,000 than to $2,000.

This is the other big issue in our industry: how do we protect the assets that we shot for 5, 10, 20 years into the future? If you are a major studio, money is no object and there are many solutions. However, if you are an independent producer, or small production company, dollars are hard to come by. There are no good archiving solutions that are reasonably priced.

I spoke with the three founders of Ultrium, the consortium of HP, IBM, and Quantum that invented LTO, about when they expect to provide Thunderbolt-based LTO storage? All three said that they had nothing to announce and the consortium did not have a position on how devices connect to computers.
Again, we could take existing gear - currently costing $7,000 - 9,000 and use Thunderbolt converter boxes to connect it to an iMac, but this simply takes a unit, which is already too expensive and makes it even more unaffordable.

[NOTE: The Tolis Group announced yesterday new gear aimed at creative producers. The ArGest line supports both LTO-5 and LTO-6, and the Thunderbolt version, which still requires a converter box, starts at $6,898. (Information about this new product is not yet on theirWebsite.)
I've said this before and I'll say it again: The LTO vendor that can figure how to provide a direct-attached LTO drive that works with a Mac and connects directly via Thunderbolt for less than $4,000 is going to make a lot of money.

For now, I really need some way to archive my media. But none of the units out there support either my budget or my computer.

SUMMARY
Buying any computer is always a trade-off between dreams, performance and budget. I'm looking forward to getting my new system. I'm also looking forward to figuring out what I can use for external storage. To me, that is the key to successful video editing - storage that is large, fast, secure and affordable. And some way to back it all up.


Larry Jordan is a producer, director, editor, author, and Apple Certified Trainer with more than 35 year's experience. Based in Los Angeles, he's a member of the Directors Guild of America and the Producers Guild of America. Visit his Website at http://www.larryjordan.biz.


Continue reading "Configuring an iMac for video editing" »

Permalink | Comments(0)