Import Big CSV

[happydg]

Hello,

I would like to import big csv files with a lot of galleries ( like 600k galleries ), but browser times out before ending. Is there a way to do it server side and not on the browser?

Thank you.

[admin]

Hi !

http://smartcj.com/wiki/doku.php?id=new ... _large_cvs

[happydg]

1) What Period, hours should i set if i have 5 sets adding galleries so it doesn't overwhelm my dedicated server?

2) I added a big csv file ( 200 mo ) and after clicking on test it says: "This source is too big (bigger then 600kb) to be parsed at a regular pace. You can add it, BUT it will be parsed ONE TIME ONLY. Ie it will NOT be parsed every X hours."

So it won't parse 500 new videos per hour like cut.php should do?

Thank you.

[admin]

1. you can set 1, should be fine anyway

2. did you add it with &test=true as wiki says ?

[happydg]

2. I forgot to add it, my mistake.

[admin]

ok

[happydg]

Even with &test=true i get:

"This source is too big (bigger then 600kb) to be parsed at a regular pace. You can add it, BUT it will be parsed ONE TIME ONLY. Ie it will NOT be parsed every X hours."

I sent you the link via pm.

Thank you.

[admin]

What url exactly are you trying to add ?

[admin]

Usually that message pops up when import part is bigger then 600k

So probably 500 lines are too much, try to lower to to 200 for rexample

[happydg]

I imported the xhamster.com big dump. It's 1.8 gb and has millions of galleries.

I used this way to import it http://smartcj.com/wiki/doku.php?id=new ... _large_cvs with 150 galleries per hour.

It just imported 11k galleries and the xhamster.csv file is now at 0kb