Import Big CSV

admin
Site Admin
Posts: 37249
Joined: Wed Sep 10, 2008 11:43 am

Re: Import Big CSV

Post by admin »

I think you cut most of it testing w\o &test parameter
Try again plz to see if the issue persists.
Don't forget to run script update
happydg
Posts: 412
Joined: Mon Jul 08, 2013 2:22 pm

Re: Import Big CSV

Post by happydg »

1) If i readd the same csv which already added galleries, will it check for duplicate galleries in sets?

2) Do i need to create a cut.php and tmp.file per csv or i can use the same for all csv files?

Thank you.
admin
Site Admin
Posts: 37249
Joined: Wed Sep 10, 2008 11:43 am

Re: Import Big CSV

Post by admin »

1. yes
2. You can use the same
Don't forget to run script update
happydg
Posts: 412
Joined: Mon Jul 08, 2013 2:22 pm

Re: Import Big CSV

Post by happydg »

In rotation - import, how to set more time before it times out after importing gals?
admin
Site Admin
Posts: 37249
Joined: Wed Sep 10, 2008 11:43 am

Re: Import Big CSV

Post by admin »

No way, it depends on browser
Add it as importset
Don't forget to run script update
happydg
Posts: 412
Joined: Mon Jul 08, 2013 2:22 pm

Re: Import Big CSV

Post by happydg »

I did multiple tests with this script.

1) It's not a good idea to use the same tmp.file and cut.php for all sets. At some point, it will mix galleries and you will end up with xhamster galleries in pornsharing csv...

2) This script has a bug. For example i had a dump with 60K galleries. This dump went to 0kb ( the tmp file also ) but only imported 20k galleries. About the same problem with xhamster dump. It magically went from 1500mb to 29mb dump file in one update set but when i check my xhamster galleries it only has 70k galleries. At this point the dump file should be at at least 1300mb.
admin
Site Admin
Posts: 37249
Joined: Wed Sep 10, 2008 11:43 am

Re: Import Big CSV

Post by admin »

1. yes, a new feature was added to the nest version to import large bumps w\u cut.php

2. The script is just 5 lines long and really easy to understand I think ..
There's no place to make a mistake
most likely you\some other scripts\whatever runs cut.php at the same time
Don't forget to run script update
happydg
Posts: 412
Joined: Mon Jul 08, 2013 2:22 pm

Re: Import Big CSV

Post by happydg »

1. I updated script to latest version. Could you please let me know where this new feature is?

Thank you.
admin
Site Admin
Posts: 37249
Joined: Wed Sep 10, 2008 11:43 am

Re: Import Big CSV

Post by admin »

Last version is branch 2, not 1.x
Don't forget to run script update
happydg
Posts: 412
Joined: Mon Jul 08, 2013 2:22 pm

Re: Import Big CSV

Post by happydg »

What ssh command to run this upgrade?
Post Reply