Emitter class reference

Emitters are used to export models to output format.

This module contains all classes for emitters: base and actuals. Currently the system has two emitters: CSVEmitter and MySQLEmitter implemented, of which the last is the default emitter. An emitter provides the export format for the scanned and cleaned datasets. It also provides preambles and postambles in the output files, for example to clean the target table before loading it.

The following classes are defined in this module:

The basic structure for emitting is a combination between BaseManager and BaseEmitter:

e = Emitter(manager=Model.objects)
print e.preamble(header=[..my header lines to add..])
for l in Model.objects.all():
  print e.emit(l)  # emit is returning a list of strings!

Note

At this moment data-migrator does not an actively take part in schema migrations of any sort. It is purely about cleaning, anonymizing and transforming data (yet!).

MySQLEmitter

class data_migrator.emitters.mysql.MySQLEmitter(*args, **kwargs)[source]

MySQL emitter to output MySQL specific insert statements

base_template

base template to output the object

extension

file extension for output file of this emitter. Defaults to .sql

Type:str
emit(l)[source]

Output the result set of an object as MYSQL insert statement

preamble(headers)[source]

override the preamble method to make it specific for MySQL

CSVEmitter

class data_migrator.emitters.csv.CSVEmitter(*args, **kwargs)[source]

CSV emitter to output delimited data

base_template

base template to output the object

extension

file extension for output file of this emitter. Defaults to .csv

Type:str
emit(l)[source]

Output the result set of an object as CSV string

preamble(headers=None)[source]

generate a premable for the file to emit.

Parameters:headers (list) – additional header to provide outside the emitter (e.g. statistics)
Returns:preamble lines
Return type:list

JSONEmitter

class data_migrator.emitters.json_emit.JSONEmitter(*args, **kwargs)[source]

JSON emitter to output as JSON Messages

extension

file extension for output file of this emitter. Defaults to .json

Type:str
emit(l)[source]

Output the result set of an object as JSON dump

preamble(headers=None)[source]

generate a premable for the file to emit.

Parameters:headers (list) – additional header to provide outside the emitter (e.g. statistics)
Returns:preamble lines
Return type:list

UpdateEmitter

class data_migrator.emitters.update.UpdateEmitter(*args, **kwargs)[source]

MySQL emitter to output MySQL specific update statements

base_template

base template to output the object

extension

file extension for output file of this emitter. Defaults to .sql

Type:str

Note: at least on field needs to have key=True

emit(l)[source]

Output the result set of an object as MYSQL insert statement

preamble(headers)[source]

override the preamble method to make it specific for MySQL

BaseEmitter

class data_migrator.emitters.base.BaseEmitter(extension=None, manager=None)[source]

Base for emitters of the data-migrator.

manager

reference to the manager that is calling this emitter to export objects from that manager

Type:BaseManager
model_class

reference to the model linked to the class

Type:Model
extension

file extension for output file of this emitter

Type:str

note: model_class and manager are linked together

__init__(extension=None, manager=None)[source]

x.__init__(…) initializes x; see help(type(x)) for signature