tesser.hadoop.serialization

Lets us serialize Tesser reduction state for transport through various distributed systems

fress-handlers

All the serialization you could ever need in one fat monolithic, hmm, I dunno, is this really such a good idea?

handler

macro

(handler classname tag write-expr read-expr)

Takes a classname as a symbol, a tag name as a string, and bodies for write and read functions. Provides a special syntax for writing the component count: (write-tag! some-number), which expands to (.writeTag writer tag some-number). Returns a map with two keys: :readers, and :writers, each value being a map suitable for use as a Fressian reader or writer, respectively.

(handler QDigest "q-digest"
  (write [_ writer digest]
    (write-tag! 1)
    (.writeBytes writer (QDigest/serialize digest)))
  (read [_ reader tag component-count]
    (QDigest/deserialize ^bytes (.readObject reader))))

handlers

macro

(handlers & quartets)

Takes a flat series of handler quartets: class-name, tag, writer, reader, as per handler. Returns a {:writers {...}, :readers {...}} map, where all writers are merged into a unified map, merged with the clojure default handlers, and wrapped with inheritance/associative lookups. Does the same for the readers map, but without inheritance lookups. :readers and :writers may be passed to Fressian.

(handlers
  QDigest "q-digest"
  (write [_ writer digest]
    (write-tag! 1)
    (.writeBytes writer (QDigest/serialize digest)))
  (read [_ reader tag component-count]
    (QDigest/deserialize ^bytes (.readObject reader)))

  clojure.lang.PersistentVector "vector"
  (write [_ writer v]
    (write-tag! (count v))
    (doseq [e v]
      (.writeObject writer e)))
  (read [_ rdr tag component-count]
        (let [v (transient [])]
          (dotimes [_ component-count]
            (conj! v (.readObject rdr)))
          (persistent! v))))

read-byte-array

(read-byte-array bs)

Parse a byte array and return a single object, using our handlers.

rewind

(rewind buf)

Rewind a byte buffer, returning it.

write-byte-array

(write-byte-array x)

Dump a structure to a byte array using our handlers.