Serialisation

cod does message oriented communication. Messages are entirely defined by serializers.

Marshalling your objects

The default serializer in cod uses Ruby object marshaling as a form of on-the-wire communication that is both efficient and fast. The downside is that you cannot communicate with other versions of Ruby.

It is, on the other hand, nice to be able to transmit almost any Ruby object, with only a few exceptions:

Procs and Lambdas
File descriptors (with some exceptions, see below)

cod has a mechanism built in where you can transmit cod channels via almost any other cod channel. This mechanism mostly does what you’d expect - give you access to a channel on the other side. Some cases don’t work yet - and will be extended as the need arises.

So this works:


  read, write = Cod.pipe.split
  write.put read
  
  # Transmitted the read-end of the channel itself!
  read.get.object_id == read.object_id   # => true

Roll your own

Let’s have a look at how the default serializer is implemented:


  class DefaultSerializer
    def en(obj)         # (1)
      Marshal.dump(obj)
    end

    def de(io)          # (2)
      if block_given?   # (3)
        Marshal.load(io, Proc.new)
      else
        Marshal.load(io)
      end
    end
  end

Message encoding (1) is really simple. You get the message to encode passed as parameter and you return a string that should be sent through the communication channel. You have to worry about delimiting the message yourself.

Message decoding (2) operates directly on the channels IO stream. It should read one message and return the message object to its caller.

If a block is given (3) to the decoding mechanism, it should call that block for each object that it decodes. The block then returns the object to insert into the decoded object graph. This is used for turning on-the-wire versions of objects that cannot otherwise be serialized into a corresponding object.

More efficient: Protobufs

As a proof of concept, cod also contains a (google) protobuf serializer that transmits its messages as protobuf objects. The format chosen here is generic – class name is transmitted along with the actual object data. This method can be used to transmit any protobuf object.

If you have only a few objects to transmit, I suggest you roll your own serializer. Once you arrange your definitions so that you don’t have to transmit lengths and class name, communication can be very efficient.

protobuf serialization is also useful for communication with servers NOT written in Ruby.

Here’s how you use the protobuf serializer in your own code:


  # gem install protobuf...
  
  serializer = Cod::ProtocolBuffersSerializer.new
  
  pipe = Cod.pipe(serializer)
  tcp  = Cod.tcp('foo:1234', serializer)
  # .. and so on.

Text protocols

Cod also implements various text protocols. This means that line ends delimit messages and that only strings can be transmitted.

For simple line by line serialization you can use LineSerializer.


  out = Cod.stdio(Cod::LineSerializer.new)
  
  out.put 'holler'    # puts 'holler' to stdout

Cod.beanstalk internally uses a serializer called Cod::Beanstalk::Serializer, which is also line oriented.

Other things to look at

panzer is a web server using cod. The whole HTTP protocol implementation lives inside a cod serializer, which turns HTTP traffic into a rack compatible request with the aid of net-http-server. Ok, now I have a headache.