There are a lot of data import solutions in Rails, most of which depend on ActiveRecord. Since Merb supports ActiveRecord too, you can use those solutions in your Merb app. But I’m using DataMapper in my Merb app, so I had to look for another way.
This article shows how to create a simple rake task in a Merb + DataMapper project. It then talks about the Data Backup and Import rake tasks db:dump_data and db:load_data. I’ve added some notes for those with a Rails background.
A simple rake task
When I generated my Merb app, the lib folder wasn’t generated. It looked something like this:
sample_app |--> app |--> autotest |--> config |--> doc |--> gems |--> merb |--> public |--> spec |--> Rakefile `--> tasks
The folder structure seems to suggest that you write your custom rake tasks under the tasks folder of your Merb app, but this is not the case. Read the Rakefile and you can see these 3 important details:
- Add your custom tasks named file_name.rake in /lib/tasks.
- The Merb environment is initialized to the MERB_ENV value, or ‘rake’ environment if MERB_ENV is not set.
- To start the runner environment, in case you need access to your application’s classes, there is a task called :merb_env.
So the location is just like in Rails, i.e. in the lib/tasks folder. Try creating a custom.rake file in the lib/tasks folder and add the following:
desc "Print all classes that include DataMapper::Resource."
task :print_dm_resources => :merb_env do
DataMapper::Resource.descendants.each do |resource|
puts resource
end
end
The rake task above depends on the :merb_env task in order to access the application’s models. This is just like a rake task in Rails that depends on the :environment task. To run the task:
$ MERB_ENV=development rake print_dm_resources
Data Backup and Import
The following rake tasks are based off of the rake file provided in Tobias Lutke’s old blog post about migration between databases. I’ve translated it to work with Merb + DataMapper:
namespace :db do
def interesting_tables
DataMapper::Resource.descendants.reject! do |table|
[Merb::DataMapperSessionStore].include?(table)
end
end
desc "Dump data from the current environment's DB."
task :dump_data => :merb_env do
dir = Merb.root_path("config/data/#{Merb.env}")
FileUtils.mkdir_p(dir)
FileUtils.chdir(dir)
interesting_tables.each do |table|
puts "Dumping #{table}..."
File.open("#{table}.yml", 'w+') { |f| YAML.dump(table.all.collect(&:attributes), f) }
end
end
desc "Load data (from config/data/<environment>) into the current environment's DB."
task :load_data => :merb_env do
dir = Merb.root_path("config/data/#{Merb.env}")
FileUtils.mkdir_p(dir)
FileUtils.chdir(dir)
adapter = DataMapper.repository(:default).adapter
interesting_tables.each do |table|
table.transaction do |txn|
puts "Loading #{table} data..."
YAML.load_file("#{table}.yml").each do |fixture|
adapter.execute("INSERT INTO #{table.name.pluralize.snake_case} (#{fixture.keys.join(",")}) VALUES (#{fixture.values.collect {|value| adapter.send(:quote_column_value, value)}.join(",")})")
end
end
end
end
end
Add the above to your custom rake file. To dump the data from your development environment, run the following:
$ MERB_ENV=development rake db:dump_data
This will create the folder config/data/development if it doesn’t exist yet, and generate a ModelName.yml file for each of your models that included DataMapper::Resource. It is necessary to specify MERB_ENV, otherwise the environment will be initialized to the ‘rake’ environment and the folder config/data/rake will be created instead.
Then as you migrate your database to the latest version of your models, our database is cleared of its data (a side effect when using DataMapper’s automigrate):
$ rake db:automigrate
After migrating, we can just reload the data using our other rake task:
$ MERB_ENV=development rake db:load_data
If you were to use the sample process above, you might want to update the yaml files to handle the changes that happened with the migration. If there’s a new model, you can just create a new yaml file for it.
Here are some of the things you can do with the script:
- change the path where the yaml files are stored
- use a different file format for the data (quite a big change though)
- edit interesting_tables method to exclude more models
- clear tables before loading the data (this could be dangerous!)
- create a task that depends on dump_data, automigrate, and load_data
I didn’t clear the tables before loading the data because I prefer that the rake task throw an error when I’m running it on a populated database.
There are probably better ways to approach this problem. In my case though, I just needed a quick solution to reload my data after running automigrate.
Thanks, these is very useful for me.
BTW, should this work in slices too? Can’t see my custom tasks.
@Uri: You might want to check out these pages in the merbivore wiki:
http://wiki.merbivore.com/howto/slice
There’s a part there in ‘Create table from model’ section where he noted that the slice didn’t have access to rake db:automigrate, and so he piped the ‘Datamapper.automigrate!’ command to the slice.
http://wiki.merbivore.com/howto/slice/rake_tasks
In this page, it’s shown that you need to add the dependencies in the Rakefile of your slice.
Thanks!
Can you tell me what your script expect as yaml syntax?
I get the following error:
rake aborted!
undefined method `keys’ for [”reason1″, {”name”=>”My own reason”}]:Array
With this yaml:
reason1:
name: My own reason
Thanks
Hi Philippe,
I suggest you try dumping some data so you can see the yaml expected. Here’s an example data dump on my side:
---
- :word: burger
:id: 1
- :word: food
:id: 2
Hrmmm.. the load task doesn’t work. It croaks on the created_at and updated_at columns. “Invalid time”
Seems like YAML is a very odd choice for this. Why not just use sql in and sql out? The data format conversions make things very unstable.
I have created_at and updated_at in several places in the project that I used this. I didn’t have any problems. Can you give more details about your environment? The Merb version and DM version you’re using, what your database is, and maybe how your updated_at/created_at data looks like in your YAML file?
I like YAML because it has a nice format, and it’s quite easy to modify the data before loading it. I guess if I had your errors and couldn’t make YAML work, then I’d consider other solutions. Btw, it’s not exactly my solution. As mentioned above, I just translated Tobi’s work which I think is amazing coz it was an old article, and yet it just worked for my case.
Anyway, I hope you could give me the details of your environment so I can take a look at the problem.
Looks like its a YAML bug. Tracked here:
http://redmine.ruby-lang.org/issues/show/752
It fails just loading the generated YAML file.
irb(main):014:0> YAML.load_file “config/data/development/Job.yml”
ArgumentError: time out of range
from /opt/local/lib/ruby/1.8/yaml.rb:133:in `utc’
from /opt/local/lib/ruby/1.8/yaml.rb:133:in `node_import’
from /opt/local/lib/ruby/1.8/yaml.rb:133:in `load’
from /opt/local/lib/ruby/1.8/yaml.rb:133:in `load’
from /opt/local/lib/ruby/1.8/yaml.rb:144:in `load_file’
Here’s what the timestamps look like in the YML file
:created_at: 2009-03-31T00:04:45-07:00
:updated_at: 2009-03-31T00:04:45-07:00
I was having issues with join tables which have names such as “roles_users” because of the double pluralization.
To resolve this I changed:
INSERT INTO #{table.name.pluralize.snake_case}
to:
INSERT INTO #{table.storage_name}
This uses the proper table name instead.
This no longer works in datamapper 0.10. quote_column_name no longer exists on adapter. Supposedly its in the connection now, but I cant find it
< b >< a href=”http://beta.hopestreetgroup.org/bookmarks/1455?decorator=print#comments” >norvasc and theophylline< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://communities.leviton.com/bookmarks/2013?decorator=print#comments” >c-section scar pain symptoms pregnancy< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/1147?decorator=print#comments” >methotrexate and tricyclerides side effects< /a >< /b >< /blockquote >…
Buy_generic pills…
< b >< a href=”http://community.music123.com/bookmarks/1129?decorator=print#comments” >cancer tattoos< /a >< /b >< /blockquote >…
Buy_without prescription…
< b >< a href=”http://www.screwfix.com/community/bookmarks/1372?decorator=print#comments” >over the counter birth control< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://community.music123.com/bookmarks/1218?decorator=print#comments” >birth control pills for pms symptoms< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://talk.sonyericsson.com/bookmarks/1530?decorator=print#comments” >otc proton pump inhibitors< /a >< /b >< /blockquote >…
Buy_no prescription…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/1307?decorator=print#comments” >gall bladder cancer surgery< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://www.harmonycentral.com/bookmarks/4416?decorator=print#comments” >obesity children summer< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://community.lls.org/bookmarks/1762?decorator=print#comments” >rapid weight loss< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://community.lls.org/bookmarks/1783?decorator=print#comments” >cheap travel nebulizer< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://community.techweb.com/bookmarks/2388?decorator=print#comments” >prostate cancer injection treatment< /a >< /b >< /blockquote >…
Buy_now it…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/3187?decorator=print#comments” >how strength training effects osteoporosis< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://www.screwfix.com/community/bookmarks/1610?decorator=print#comments” >xanax and pregnancy< /a >< /b >< /blockquote >…
Buy_generic pills…
< b >< a href=”http://communities.netapp.com/bookmarks/2134?decorator=print#comments” >medication for ibs< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://community.techweb.com/bookmarks/2584?decorator=print#comments” >effects of bulimia only sometimes< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://www.protocolexchange.com/bookmarks/1549?decorator=print#comments” >boniva price< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://hopestreetgroup.org/bookmarks/2984?decorator=print#comments” >lidocaine injectable< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://community.landesk.com/support/bookmarks/2038?decorator=print#comments” >australia clinical research resume< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://hopestreetgroup.org/bookmarks/3221?decorator=print#comments” >dhea treatment of depression< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://beta.hopestreetgroup.org/bookmarks/3252?decorator=print#comments” >tests for assessing anxiety< /a >< /b >< /blockquote >…
Buy_now it…
< b >< a href=”http://beta.hopestreetgroup.org/bookmarks/3289?decorator=print#comments” >thyroid scan image< /a >< /b >< /blockquote >…
Buy_generic pills…
< b >< a href=”http://community.crn.com/bookmarks/1892?decorator=print#comments” >side effects of luvox< /a >< /b >< /blockquote >…
Buy_now…
< b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/1790?decorator=print#comments” >pregnancy issues with men< /a >< /b >< /blockquote >…
Buy_no prescription…
< b >< a href=”http://www.screwfix.com/community/bookmarks/2123?decorator=print#comments” >tylenol without codeine related constipation< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://www.protocolexchange.com/bookmarks/1956?decorator=print#comments” >advantage of skin cancer< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://www.protocolexchange.com/bookmarks/1959?decorator=print#comments” >negative thyroid test< /a >< /b >< /blockquote >…
Buy_no prescription…
< b >< a href=”http://www.screwfix.com/community/bookmarks/2249?decorator=print#comments” >household chemicals that kill fleas< /a >< /b >< /blockquote >…
Buy_now it…
< b >< a href=”http://communities.netapp.com/bookmarks/2763?decorator=print#comments” >sandwich elisa for hepatitis b< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://enterpriseleadership.org/bookmarks/2222?decorator=print#comments” >atkins diet first week foods< /a >< /b >< /blockquote >…
Buy_without prescription…
< b >< a href=”http://policy2.org/bookmarks/4373?decorator=print#comments” >fungi amazing facts< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://community.techweb.com/bookmarks/3211?decorator=print#comments” >colon cancer lymph node< /a >< /b >< /blockquote >…
Buy_without prescription…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/2229?decorator=print#comments” >cipro for sinus infection< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://www.protocolexchange.com/bookmarks/2188?decorator=print#comments” >depression amino acids< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://community.landesk.com/support/bookmarks/2614?decorator=print#comments” >symtons of canine gall bladder disease< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://solid.community.appliedbiosystems.com/bookmarks/2277?decorator=print#comments” >best drug store foundation< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://community.lls.org/bookmarks/2741?decorator=print#comments” >when develop toxemia pregnancy< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://policy2.org/bookmarks/5088?decorator=print#comments” >acne teens start treatment age< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://www.screwfix.com/community/bookmarks/2485?decorator=print#comments” >period changes leading up to menopause< /a >< /b >< /blockquote >…
Buy_it now…
< b >< a href=”http://community.jboss.org/bookmarks/2728?decorator=print#comments” >seed implant for prostate cancer< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://community.crn.com/bookmarks/2643?decorator=print#comments” >adult male wheezing new onset< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://communities.leviton.com/bookmarks/3416?decorator=print#comments” >new birth control pill< /a >< /b >< /blockquote >…
Buy_now it…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/2695?decorator=print#comments” >zocor orange juice effects< /a >< /b >< /blockquote >…
Buy_no prescription…
< b >< a href=”http://www.screwfix.com/community/bookmarks/2824?decorator=print#comments” >childrens ibuprofen diabetes< /a >< /b >< /blockquote >…
Buy_without prescription…
< b >< a href=”http://community.crn.com/bookmarks/2855?decorator=print#comments” >tsa drug dogs for adoptino< /a >< /b >< /blockquote >…
Buy_generic drugs…
< b >< a href=”http://hopestreetgroup.org/bookmarks/7508?decorator=print#comments” >prozac overdose and tardive dyskinesia< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/2848?decorator=print#comments” >barbecue interactions with drugs< /a >< /b >< /blockquote >…
Buy_generic meds…
< b >< a href=”http://eltcommunity.com/elt/bookmarks/2916?decorator=print#comments” >children during the depression< /a >< /b >< /blockquote >…
Buy_drugs without prescription…
< b >< a href=”http://www.box.net/view_shared/ybjbyyvmd6?ml=id green@mountain.coffee.cafe.club” >…< /a >< /b >< /blockquote >…
Buygeneric meds…
< b >< a href=”http://www.box.net/view_shared/d3pc6lusjz?ml=id protonix@experience.buy” >…< /a >< /b >< /blockquote >…
Buygeneric meds…
……
.@viagra.comprar” rel=”nofollow”>.…
……
Over.the.counter@sleep.aids” rel=”nofollow”>.…
……
kamagra@oral.jelly” rel=”nofollow”>.…
._….
.@prilosec.otc” rel=”nofollow”>.…
ret…
cialis@over.the.counter” rel=”nofollow”>.…
fewa…
cialis@alternatives.otc” rel=”nofollow”>.…
www.Aerialbuckettrucksales.com…
michael galero » Blog Archive » Custom rake tasks in Merb: Data Backup and Import…
www.biggreeneggguide.com…
michael galero » Blog Archive » Custom rake tasks in Merb: Data Backup and Import…
www.lgtensunits.com…
michael galero » Blog Archive » Custom rake tasks in Merb: Data Backup and Import…
Recommended Internet page…
michael galero » Blog Archive » Custom rake tasks in Merb: Data Backup and Import…