Nando's blog
About Nando Nando's blog Posts about video, movies etc. Posts about computing Posts about music Posts about literature Philosophical posts Posts about programming

I dream of a db4o for Python

I wish Python would have an object database that wouldn’t eat up all the RAM.

Such a project could also be defined as “Durus with queries that don’t have to first instantiate objects”.

It would need a custom file format, without using Python’s pickles, to be able to query without instantiating. And I wouldn’t mind losing some dynamic-language-like flexibility if I could have this.

I have been using the SQLAlchemy ORM because it uses very little memory, it is very powerful and very fast. It does everything you need and more. However...

For some projects, I feel the difficulties in defining SQLALchemy models (and dealing with the relational database behind it). I want to use Python types, not relational database types. I would switch to an object database, if only it would query and use little RAM.

(In Python, making a tuple is faster than making a list is faster than making a dict is MUCH faster than instantiating an object.)

If I were to start such a db4o-like project (maybe 2010?) I would have to wrap my head around pointers to objects and their activation. So at first I might use SQLAlchemy as a backend. A table of properties... belongs to a table of objects... which know their Python type. This would probably enable one to make those queries without first instantiating.

About this scheme, ronny tells me to take a look at RDF – neat for dealing with not clearly defined object graphs.

(This is not my area of expertise...)

If you are a real good, but real good Python developer, go ahead and do it! But heads up, you’re gonna face DAMN tricky stuff about reference cycles and efficient collections.

This project is for the future. Python 3.0 on it!