full base dump questions

[oceanude]

Hi,

Since 48h I'm running an xhamster full basedump.
I set cut.php to 5000 and interval to 1h, grabbing all 10 thumbs/video, just copying them, not cropping/resizing.
It's grabbing at appr. 35 videos/minute = 50,000 videos/day, so it will take 1 month to complete this import.

Is there any way to speed up this import?

I'm also getting some error message, I don't know if they're crucial:
- preload log:
"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 144385281 bytes) in /usr/.../public_html/scj/admin/files/logs.php on line 0 err"
- error.log:
50 messages/minute: "04.28 SELECT gallery_slug FROM rot_gallery_data1 WHERE gallery_md5 = '4760248b4dacfc080867015ded288c67' :: : "
- 2014-04-15 09:08:50: Allowed memory size of 134217728 bytes exhausted (tried to allocate 123908869 bytes) in /usr/.../public_html/scj/admin/files/logs.php on 0
- 2014-04-15 09:12:49: Allowed memory size of 134217728 bytes exhausted (tried to allocate 125022392 bytes) in /usr/.../public_html/scj/admin/files/logs.php on 0

Thanks.

[admin]

1. did you run update?
2. It won't work this way, don't be greedy )
Hamster DB is about 4M movies * 10 thumbs = 40M thumbs , script won't be able to rotate it normally

[oceanude]

1. Yes, V 1.50 - Build 201402240856

2. But it will rotate normally with "only" 4M * 1 thumb ?
I wanted to have all 40M thumbs in a DB so I would never have to go add a thumb manually if I find one somewhere else that isn't standard no1 thumb by xhamster. Not rotating the whole 40M actively.

[admin]

Well, in theory it can, but we didn't test it this way.
Feel free to try, but you'll need really powerful server
And yes, it'll take a long time to import 40M