Video encoding with Encoding.com and Rackspace Cloudfiles

We have been working with a couple of media centric web-portals which require video encoding, storage and streaming. Initial guidance and research showed us a couple of successful approaches:

• Heywatch! with Amazon S3 / Amazon CloudFront
• encoding.com with S3 / Cloudfiles /
• Limelight CDN solution

After some deliberation and talks we finally settled on encoding.com with Cloudfiles CDN. We wanted the following requirements to be met before finalizing on something:

• faster video streaming. We evaluated ooyala.com for continuous streaming but it was pretty expensive. Finally settled on the longtail flash player.
• mp4 support to ensure we are HTML5 compatible later. We thought about encoding to flv but finally dropped it altogether — lets live in the real world folks!
• Simple solution with longtail provided us an easy player, a cool skin and potential for future continuous streaming.
• Rackspace CDN seemed a good solution to ensure faster downloads!

Encoding.com took preference over heywatch for some of the following reasons: (something heywatch may want to incorporate):

• Support for cloudfiles. Heywatch integrates seamlessly with S3 but does not support cloudfiles (when I last checked!)
• Notifiy_url – the URL that should be called after an encoding task is complete is very easily configured in the encoding.com task itself! On Heywatch, we have to configure their system. This brings on a few limitations — on one callback URL. In encoding.com, we can pass the notify_url as a parameter, so its far more customizable.
• encoding.com can support multiple sub-tasks inside one task. A task has a media-id and its sub-tasks have task-ids. Heywatch does not support sub-tasks. So, if I want to encode a video in 3 different formats, we have to fire 3 tasks on heywatch but 1 task in encoding.com. I get one callback for all sub-tasks instead of waiting on 3 tasks from heywatch.

I found the encoding.com sub-tasks support most helpful as I wanted to encode a video in 2 different sizes, different formats and also wanted to first frame image from the video for a thumbnail. All these were 1 media-task for encoding and it returns me ONE callback after all the tasks are complete. If I had used Heywatch, I would have to pass some ‘captive data’ in each task request and query them back in the response and map the task completion — painful!

My previous post detailed using CloudFiles and paperclip. Here I shall detail out how to use encoding.com and store encoded videos onto ClouFiles. The encoding.com site points to Ruby library for using encoding.com API. Its very very primitive and I strongly recommend installing the encoding-dot-com plugin. In fact, I believe encoding.com should point people to this plugin!

$ ./script/plugin install http://github.com/esp/encoding-dot-com

Configure your initializer so that you have access to the encoding initialized object:

# config/initializers/encodingdotcom
ENCQ = EncodingDotCom::Queue.new(<userid>, "<secret>")

To start any encoding job, all you have to do is invoke it !

mediaid = ENCQ.add_and_process(
    resource.container_url, #source
    {
        # Task 1: Encode into a 608x size video (normal)
        resource.container_encoded_url => EncodingDotCom::Format.create(
             'output' => 'mp4',
             'size' => '608x0',
             'add_meta' => 'yes',
             'bitrate' => '1024k',
             'framerate' => '25',
             'video_codec' => 'libx264',
             'audio_bitrate' => '128k',
             'profile' => 'baseline',
             'two_pass' => 'yes'),
        # Task 2: Encode into a 320x size video (preview)
        resource.container_encoded_url('preview') => EncodingDotCom::Format.create(
             'output' => 'mp4',
             'size' => '320x0',
             'add_meta' => 'yes',
             'bitrate' => '1024k',
             'framerate' => '25',
             'video_codec' => 'libx264',
             'audio_bitrate' => '128k',
             'profile' => 'baseline',
             'two_pass' => 'yes'),
        # Task 3: Generate a thumbnail
        '' => EncodingDotCom::Format.create('output' => 'thumbnail'),
     },
     { 'notify' => "http://someserver.com/callback" }
     # Alternatively you could also:
     #{ 'notify' => "someone@somewhere.com" }
)

Now, when the task and its sub-tasks complete, encoding.com shall callback the URL you had specified. The response is in XML:

<?xml version="1.0"?>
<result>
 <mediaid>1299863</mediaid>
 <source>source-file-url</source>
 <status>Finished</status>
 <description></description>
 <format>
  <taskid>5590432</taskid>
  <output>thumbnail</output>
  <status>Finished</status>
  <destination>destination-file-url-for-image</destination>
 </format>
 <format>
  <taskid>5590433</taskid>
  <output>mp4</output>
  <status>Finished</status>
  <destination>destination-file-url-for-mp4</destination>
 </format>
 <format>
  <taskid>5590434</taskid>
  <output>flv</output>
  <status>Finished</status>
  <destination>destination-file-url-for-flv</destination>
 </format>
</result>

Parsing this is easy via Nokogiri. Ensure that the media-task has status ‘Finished’ and not ‘Processing’ or ‘Error’. I used the following code for checking on response. (Assume the the notify_url specified was http://myserver/controller/callback

# controller_method for callback processing
def callback
  xml = Nokogiri::XML(params['xml'])
  mediaid = xml.xpath('/result/mediaid').text

  # Check the status of the tasks are 'Finished'.
  xml.xpath('/result/format/status').each do |status|
    if status.text == 'Error'
      #Do your error processing
    end
  end
end

Done!

The beauty of this approach was that we used paperclip to upload the file to the source destination, then used encoding.com to encode the videos into the desired format (inclduing getting the thumbnail from the first frame).

Enhancement: I am now trying out the nginx upload module to upload files to make uploads faster.