Friday, October 19, 2012

[solved] Move django from SQLite to MySQL

When you start develop new site, you may run from little one with tiny database.
For django best start from sqlite and after some time, if you project begin grow you move to more serious database engine (MySQL, Postgre SQL etc).
Sadly, django does not have any ready tools to move from one to another database.
You may try

  • python ./manage.py dumpdata > data.json

and

  • python ./manage.py loaddata data.json

but unfortunately this doesn't work because of integrity errors...or many other issues

For move from one db to another i follow steps:
  1. Edit settings.py 
    1. add new database in DATABASES settings with 'slave' name like:
      DATABASES = {'default': {...}, 'slave': {...}}
      
    2. if you use apps like south - comment them for a while
  2. Create new db (django create only new file for sqlite database type)
  3. Create tables in new db
    1. python ./manage.py syncdb --database slave
    2. enable south and repeat 3.1
  4. Create file to_slave.py in main project directory:
    • from django.contrib.contenttypes.models import ContentType
      
      def run():
      
       def do(Table):
        if Table is not None:
         table_objects = Table.objects.all()
         for i in table_objects:
          i.save(using='slave')
       
       ContentType.objects.using('slave').all().delete()
      
       for i in ContentType.objects.all():
        do(i.model_class())
      
  5. If you have any kind of signals for you models - disable them before next step
  6. run 
    1. python ./manage.py shell and type command in shell
      1. from to_slave import run
      2. run()
  7. Make 'slave' default database in you project
  8. Done






10 comments:

  1. Thanks a lot. Your post was really helpful.

    But one minor thing. Even with this, I first got integrity errors. So, instead of ContentType.objects.all(), I tweaked the order in which data is copied from various tables. Then it worked.

    ReplyDelete
    Replies
    1. As quick solution - you can run that script twice :)

      Delete
  2. One thing I'd like to ask is how should one move ManyToManyFields? Using the above method, everything works like a charm except ManyToManyFields.

    ReplyDelete
    Replies
    1. yes, there is a problem this m2m fields. I'll think about the solutions.

      Delete
  3. Here is a more manual method. Supports defining the order in which the tables are copied.

    https://gist.github.com/runekaagaard/5037369

    ReplyDelete
    Replies
    1. There a trouble with all that method.
      As Prasenjit Singh mention - ManyToManyFields are not copied at all.

      Delete
  4. I keep getting

    "django.db.utils.DatabaseError: no such table: django_content_type"

    I check mysql and table has been created

    ReplyDelete
  5. From your idea, with slight modification as following, we are able to copy data to another database with a custom Django script
    1. Use apps.get_app_configs() to retrieve all models instead of using ContentType.
    2. Sort models using sort_dependencies in dumpdata command
    3. Add method to copy tables in ManyToManyFields (The value for primary key might be different, refer to the source code for explanation)
    Please see http://codeforaliving.blogspot.com/2015/03/migrating-django-database-from-one.html for details

    ReplyDelete