In ruby, the new cool thing to do when interfacing with a C library is to use FFI. The same is true with luajit as well. Instead of writing a bunch of C code, you just declare the existing C functions you want to use, and the runtime mostly handles things for you. There’s still a bit of a mismatch, because you’re not writing in C, so I generally stack a couple layers of abstraction over the top.

FFI gives us a “raw” interface. We need to “cook” it before trying to consume it.

simple example

The very first thing we do is get the C function into our Lua code. (Similar for ruby, except they have a slightly different way of declaring functions.)

ffi.cdef[[int sqlite3_open(const char *, void **);]]

local lib = ffi.load("sqlite3")

What we have now is a lib.sqlite3_open function, but it’s unwieldy because of the out parameter for the db handle. We need to wrap it.

sqlite3ffi = { }
function sqlite3ffi.open(filename)
        local pdb = ffi.new("void *[1]", nil);
        local rv = lib.sqlite3_open(filename, pdb);
        local db = pdb[0]
        return db
end

Now there’s a sqlite3ffi table (namespace) with an easier to call open function. But the db object here is still just a pointer. It’s hard to use from Lua. Obviously, if one layer of abstraction didn’t solve our problem, the solution is more abstraction.

sqlite3 = { }
function sqlite3.open(filename)
        local dbh = sqlite3ffi.open(filename)
        if not dbh then return nil end

        local db = { }
        db.dbh = dbh
        db.close = dbclose
        ...
        return db
end

Now I’ve finally created another (the third, if you’re counting) namespace, but this one has an open function that returns a Lua object, and will have a variety of (elided) methods attached to it (or you can use a metatable). There’s no real reason the sqlite3ffi and sqlite3 namespaces couldn’t be collapsed, and just have sqlite3.open call the ffi functions directly, but I prefer to keep any amount of logic separate from the marshaling code. Basically, I first make the C API callable from Lua, then I use it to make a more native feeling API. In the case of a database like this, there’s probably going to be even more DBI type abstraction over top, to allow switching between databases, but that’s independent of the ffi binding.

Back to the raw/cooked terminology, the lib functions are raw, the sqlite3ffi function are cooked, and the sqlite3 functions are what we’re assembling from cooked API and what will eventually be presented to the consumer.

complicated example

Why do it this? When the examples get more complicated, there can be a lot of marshaling code involved. For example, after executing a statement, we have to retrieve the results.

        row = { }
        colc = sqlite3ffi.column_count(stmth)
        for i = 0, colc - 1 do
                name = sqlite3ffi.column_name(stmth, i)
                typ = sqlite3ffi.column_type(stmth, i)
                if typ == sqlite3ffi.INTEGER then
                       val = sqlite3ffi.column_int(stmth, i)
                elseif typ == sqlite3ffi.FLOAT then
                       val = sqlite3ffi.column_double(stmth, i)
                elseif typ == sqlite3ffi.TEXT then
                       val = sqlite3ffi.column_text(stmth, i)
                elseif typ == sqlite3ffi.BLOB then
                       val = sqlite3ffi.column_blob(stmth, i)
                elseif typ == sqlite3ffi.NULL then
                       val = nil
                end
                row[name] = val
                row[i + 1] = val
       end
       rows[#rows + 1] = row

This doesn’t look too bad, but that’s because a lot of the complication of calling functions like sqlite3_column_text has already been taken care of.

function sqlite3ffi.column_text(stmt, col)
        local rv = lib.sqlite3_column_text(stmt, col)
        local len = lib.sqlite3_column_bytes(stmt, col)
        return ffi.string(rv, tonumber(len))
end

Embedding code like the above function for each case of the preceding loop is messy.

adaptable

A side benefit of always interfacing with the C library through a shim (sqlite3ffi above) is if we need to switch back to plain Lua, that’s exactly the interface we’d provide with our C extension. Or we could switch to using Alien.

pinvoke

Some notes, mostly for myself for future reference. C# also provides an FFI facility called P/Invoke or platform invoke or pinvoke. One thing that requires special attention on Windows is calling convention. My luajit DLL happens to want cdecl calls, but the Windows default is stdcall. Nothing that can’t be solved with some annotations, but forgetting to include them results in mayhem.

        [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
        public delegate int lua_CFunction(IntPtr state);

        [DllImport("lua51.dll", CallingConvention = CallingConvention.Cdecl)]
        static extern void lua_pushcclosure(IntPtr state, lua_CFunction fn, int n);