Dodo the stupid editor

2015-03-07

This post continues on from my previous post [1] which ended with the following


    Not being able to use either `ed` or `sed` left me a little dissapointed,
    The c program worked but in it's current form it is very brittle and isn't
    helped by all the noise of manual error checking, this all requires far too
    much effort to debug and maintain.

    I want a more general solution.

    Ideally I want a dumbed down `ed`-like that allows editing of the file in place,
    without having to read or write any of the lines I don't care about,
    taking care of all of the noise for me,
    a domain specific language for editing a file in place.

    Hopefully details and some code will come in a later post.

This scriptable editor (language) was named dodo [3]

Elevator pitch

Dodo [3] is a simple scriptable inplace file editor

That is to say that dodo is an interpreter for a small domain specific language for making small byte-by-byte changes in place to large text files.

Dodo is a wrapper around fseek, fread and fwrite. Dodo performs all edits in place on the file, keeping no temporary copies, and not loading the contents of the file into memory. This allows for dodo to efficiently edit files far too large to fit into memory.

This also means that there is no undo, and thus I only recommend using dodo on files that you can 'reproduce', my use case being making small changes to SQL dumps (which I can reproduce again with pg_dump).

A quick example

if we have a file called example.txt with the following contents


    hello world how are you

we then run the following program:

./dodo example.txt <<EOF
    p          # print (upto) 100 bytes
    p5         # print 5 bytes ('hello')
    e/hello/   # expect string 'hello'
    b6         # goto byte 6 in file
    e/world/   # expect string 'world'
    w/marge/   # write string 'marge' (writes over 'world')
    q          # quit
EOF

which is also equivalent to:

echo 'pp5e/hello/b6e/world/w/marge/' | ./dodo example.txt

this program will output:


    'hello world how are you
    '
    'hello'

and leave example.txt with the following contents:


    hello marge how are you

The real test

The root issue dodo was designed to deal with was the one pointed out in my previous post [1]

Writing a dodo script equivalent to the one from that post [2] is the real test

and here it is:

b62
e/SET lock_timeout = 0;/
w/--/

b406
e/CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;/
w/CREATE PROCEDURAL LANGUAGE plpgsql;---------------------------/

q

which can be read as:

Closing notes

Dodo is yet another niche tool with a sharp edge, with no undo history it makes it easy to shoot yourself in the foot, but I do find it indispensable in editing large files when the usual toolset fails (ed, sed, vim).

Dodo was also quite fun to write as it is a very simple interpreter built for a single purpose.

You can find more about dodo on it's github page [3].



[1] postgres replication 93 to 82

[2] munge github source code

[3] dodo github project