How to import sets correctly?

[deeddy]

I was using this guide from the link below to import a really big dump file (something like 100,000 videos - 500+MB):
https://smartcj.com/wiki/doku.php?id=ne ... _of_thumbs

However, when I visited "Rotation - Sets" and tried to set up a new import set using your instructions, and when I clicked on Test - I got only one field (one column) for the importing pattern. Please check out the screenshot in the attachment.

How is it possible to define dump importing format? Is it possible to have more than one field in a dump file (something like: video url|embed code|title|thumb...)?
Is it a bug or I didn't figure out how to actually set up pattern correctly when importing a set?

[admin]

Oh, I have to update FAQ

Now you can add biog files as is.
What's the first line in that file? does it have | ?

[deeddy]

Are you referring to the dump file?

This is the first line and it is separated with pipelines "|" (no quotes):

Code: Select all

https://www.pornhub.com/embed/ph5e5f4cd082661|<iframe src="https://www.pornhub.com/embed/ph5e5f4cd082661" frameborder="0" height="481" width="608" scrolling="no"></iframe>|https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)1.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)2.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)3.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)4.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)5.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)6.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)7.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)8.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)9.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)10.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)11.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)12.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)13.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)
(mh=L2Rwv4oeqiEEp5kC)14.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)15.jpg,https://di.phncdn.com/videos/202003/04/289800662/original/(m=eaf8Ggaaaa)(mh=L2Rwv4oeqiEEp5kC)16.jpg|BrutalX - Kenzie Madison - Fuck stepsis instead of School|Big Ass,Blonde,Cumshot,Hardcore,Pornstar,Teen,Rough Sex,Step Fantasy|Kenzie Madison|Porn Hub

[admin]

ok, try to import it as is plz, w\o cut.php

lemme know if it works

[deeddy]

ok, try to import it as is plz, w\o cut.php

lemme know if it works

It works perfectly.

When I tried importing the set directly thru URL https://www.domain.com/dump/dump1.txt - it worked just fine and I was able to select pattern.

When I tried to do import set thru cut.php, it returns an empty page with no results. Both with and without &test do not work.

Code: Select all

https://www.domain.com/dump/cut.php?file=dump1.txt&test=true

I assume that means cut.php doesn't work correctly.

Is it possible that this was related to exec or pcntl_exec that are disabled in php.ini? The content of cut.php is the same as your sample in the instructions above.

[admin]

I think there's an issue with the setup of cut.php

As you use new version of the script - you can skip cut.php

[deeddy]

Does that mean in the new version it is possible to use URL of the dump file directly (without cut.php), even if it has like 1 million videos?
Thank you for all the help!

[admin]

Yes, that's correct

NP)

[admin]

The only limit here is if it takes more then 30 seconds to copy that URL
in this case i recommend to download that URL manually and import from local server

[deeddy]

The dump file is located on the same webserver where the website is. The file is really big and has 700k+ lines and 2.5GB in size. Will it be processed correctly without dump.php?
There's enough RAM on the server and disk loading is shorter than 30 seconds (probably up to 5 sec). Maybe the PHP memory limit needs to be increased? Please advise.