Gallery Grabber Threads

burn
Posts: 262
Joined: Wed Feb 01, 2017 11:27 am

Re: Gallery Grabber Threads

Post by burn »

Сделал крон раз в минуту.
Нагрузка немного выросла (ЛА15).
Грабится так же медленно, примерно 2 шт в минуту, хотя видно что может успеть 10 и больше.

Code: Select all

2021-06-07 21:52:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph5de9031a00444 (177759) (0.011527061462402, 0.010560035705566)
2021-06-07 21:52:00: Loading gallery https://www.pornhub.com/view_video.php?viewkey=ph5de9031a00444 (0.014121055603027, 0.0025899410247803)
2021-06-07 21:52:04: Gallery description is empty: Update  with 'XXX Indian'  (3.9778170585632, 3.9637091159821)
2021-06-07 21:52:04: Downloading files for a custom gallery (3.9948110580444, 0.016977071762085)
2021-06-07 21:52:04: Content type: 1 (3.9956610202789, 0.00084209442138672)
2021-06-07 21:52:04: Creating thumb from http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__bdf95ded7aed7ed49db4daeff45cff72__indian-red-girl-riding-on-me-with-moaning_0.jpg  (400x0) (Crop profile: 1)  (3.9971649646759, 0.0014979839324951)
2021-06-07 21:52:04: Downloading img http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__bdf95ded7aed7ed49db4daeff45cff72__indian-red-girl-riding-on-me-with-moaning_0.jpg to ../tmp/177759/tmp//319029.jpg, ref https://www.pornhub.com/view_video.php?viewkey=ph5de9031a00444  (3.9975531101227, 0.0003819465637207)
2021-06-07 21:52:05: Source Size: 21657 (4.2827250957489, 0.28517007827759)
2021-06-07 21:52:05: Dupe check   (4.2828750610352, 0.00013899803161621)
2021-06-07 21:52:05: Downloading thumb http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__bdf95ded7aed7ed49db4daeff45cff72__indian-red-girl-riding-on-me-with-moaning_0.jpg and make a copy (4.2838850021362, 0.0010061264038086)
2021-06-07 21:52:05: Identify /usr/bin//identify -verbose ../tmp/177759/tmp//319029.jpg (4.3427951335907, 0.058907032012939)
2021-06-07 21:52:05: Identify /usr/bin//identify -verbose ../tmp/177759/tmp//319029.jpg.thumb.jpg (4.5648899078369, 0.22209692001343)
2021-06-07 21:52:05: Detect img size for ../tmp/177759/tmp//319029.jpg.thumb.jpg (300 x 250) (4.6752319335938, 0.1103310585022)
2021-06-07 21:52:05: Saving thumb (346758): title:Indian Red Girl Riding On Me With Moaning, desc:, function rotation_get_thumb_path (force name: ) to folder 346/ filename 758_riding-me (4.678239107132, 0.0029919147491455)
2021-06-07 21:52:05: Saving location: local to folder ../thumbs//346/  (4.6784429550171, 0.0001981258392334)
2021-06-07 21:52:05: save_name /758_riding-me.jpg (4.6785659790039, 0.00011801719665527)
2021-06-07 21:52:05: Saving file ../tmp/177759/tmp//319029.jpg.thumb.jpg (21657 bytes) to ../thumbs//346///758_riding-me.jpg  (4.6787009239197, 0.00012993812561035)
2021-06-07 21:52:05: ../thumbs//346///758_riding-me.jpg saved (21657 bytes) (4.6790161132812, 0.00031089782714844)
2021-06-07 21:52:05: Cleanup tmp folder: ../tmp/177759 (4.6806499958038, 0.0016329288482666)
2021-06-07 21:52:05: Set gallery status to active  (4.6830620765686, 0.0024020671844482)2021-06-07 21:53:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.0011110305786133, 0.0011169910430908)
2021-06-07 21:53:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph5dbc9538521ee (177761) (0.01217794418335, 0.011062860488892)
2021-06-07 21:53:00: Loading gallery https://www.pornhub.com/view_video.php?viewkey=ph5dbc9538521ee (0.014750003814697, 0.0025689601898193)
2021-06-07 21:53:01: Not 200 answer (404)  (1.5118989944458, 1.4971528053284)
2021-06-07 21:53:01: Cleanup tmp folder: ../tmp/177761 (1.5119459629059, 3.4093856811523E-5)
2021-06-07 21:53:01: Deleting gallery (1.5122950077057, 0.00034999847412109)
2021-06-07 21:53:01: Delete gallery 177761   from /data/www/xxx-indian.net/neko/bin/gallery_grabber.php (1.5123949050903, 9.4890594482422E-5)
2021-06-07 21:53:01: Thumb deleted (1.5158038139343, 0.0034070014953613)
2021-06-07 21:53:01: Delete thumb 335806  (1.5177099704742, 0.0019028186798096)
2021-06-07 21:53:01: FS delete folder:  ../thumbs///user_thumbs/177/761  (1.5251150131226, 0.0073990821838379)
2021-06-07 21:53:01: FS_delete_folder: ../thumbs///user_thumbs/177/761 doesnt exists (1.525209903717, 9.0837478637695E-5)
2021-06-07 21:53:01: Delete gallery content 177/761 (1.5256838798523, 0.00046992301940918)2021-06-07 21:54:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.0015008449554443, 0.0015099048614502)
2021-06-07 21:54:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph5dcd6f34ab5d8 (177763) (0.014036893844604, 0.012531995773315)
2021-06-07 21:54:00: Loading gallery https://www.pornhub.com/view_video.php?viewkey=ph5dcd6f34ab5d8 (0.01708197593689, 0.0030369758605957)
2021-06-07 21:54:06: Gallery description is empty: Update  with 'XXX Indian'  (5.9932918548584, 5.9762139320374)
2021-06-07 21:54:06: Downloading files for a custom gallery (6.0060160160065, 0.012712001800537)
2021-06-07 21:54:06: Content type: 4 (6.0069139003754, 0.00088810920715332)
2021-06-07 21:54:06: Creating thumb from http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__801c84281c59f75662cc4efa58269f05__sri-lanka-new-leek-video_0.jpg  (400x0) (Crop profile: 1)  (6.0085499286652, 0.0016369819641113)
2021-06-07 21:54:06: Downloading img http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__801c84281c59f75662cc4efa58269f05__sri-lanka-new-leek-video_0.jpg to ../tmp/177763/tmp//999857.jpg, ref https://www.pornhub.com/view_video.php?viewkey=ph5dcd6f34ab5d8  (6.0088589191437, 0.00029587745666504)
2021-06-07 21:54:06: Source Size: 30285 (6.3092110157013, 0.3003568649292)
2021-06-07 21:54:06: Dupe check   (6.3093600273132, 0.00013613700866699)
2021-06-07 21:54:06: Downloading thumb http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__801c84281c59f75662cc4efa58269f05__sri-lanka-new-leek-video_0.jpg and make a copy (6.3102610111237, 0.00089597702026367)
2021-06-07 21:54:06: Identify /usr/bin//identify -verbose ../tmp/177763/tmp//999857.jpg (6.3440217971802, 0.033767938613892)
2021-06-07 21:54:06: Identify /usr/bin//identify -verbose ../tmp/177763/tmp//999857.jpg.thumb.jpg (6.3995268344879, 0.055497169494629)
2021-06-07 21:54:06: Detect img size for ../tmp/177763/tmp//999857.jpg.thumb.jpg (300 x 250) (6.4535319805145, 0.053997993469238)
2021-06-07 21:54:06: Saving thumb (346759): title:Sri lanka new leek video, desc:, function rotation_get_thumb_path (force name: ) to folder 346/ filename 759_new-leek (6.4662690162659, 0.012716054916382)
2021-06-07 21:54:06: Saving location: local to folder ../thumbs//346/  (6.4664490222931, 0.00017595291137695)
2021-06-07 21:54:06: save_name /759_new-leek.jpg (6.4665958881378, 0.00014400482177734)
2021-06-07 21:54:06: Saving file ../tmp/177763/tmp//999857.jpg.thumb.jpg (30285 bytes) to ../thumbs//346///759_new-leek.jpg  (6.4666938781738, 9.4890594482422E-5)
2021-06-07 21:54:06: ../thumbs//346///759_new-leek.jpg saved (30285 bytes) (6.4669799804688, 0.00028395652770996)
2021-06-07 21:54:06: Update flv_thumb_url for this gallery (6.4673948287964, 0.00041103363037109)
2021-06-07 21:54:06: Cleanup tmp folder: ../tmp/177763 (6.4714539051056, 0.0040559768676758)
2021-06-07 21:54:06: Set gallery status to active  (6.4801599979401, 0.0087001323699951)2021-06-07 21:54:06: processed_galleries = 2, memory = 26.7 Mb limit 512M (6.4828159809113, 0.0026531219482422)
2021-06-07 21:54:06: Processing https://www.pornhub.com/view_video.php?viewkey=ph5e21934453aed (177764) (6.4884488582611, 0.0056309700012207)
2021-06-07 21:54:06: Loading gallery https://www.pornhub.com/view_video.php?viewkey=ph5e21934453aed (6.4919009208679, 0.0034520626068115)
2021-06-07 21:54:09: Gallery description is empty: Update  with 'XXX Indian'  (9.4318268299103, 2.9399271011353)
2021-06-07 21:54:09: Downloading files for a custom gallery (9.4411249160767, 0.0092840194702148)
2021-06-07 21:54:09: Content type: 1 (9.4416708946228, 0.00054097175598145)
2021-06-07 21:54:09: Creating thumb from http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__8ba50e04eac2a55388b5de59e95af402__double-cumshot-to-neighbours-wife-horny-pussy_0.jpg  (400x0) (Crop profile: 1)  (9.4424109458923, 0.00073790550231934)
2021-06-07 21:54:09: Downloading img http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__8ba50e04eac2a55388b5de59e95af402__double-cumshot-to-neighbours-wife-horny-pussy_0.jpg to ../tmp/177764/tmp//107868.jpg, ref https://www.pornhub.com/view_video.php?viewkey=ph5e21934453aed  (9.4425239562988, 0.00010991096496582)
2021-06-07 21:54:10: Source Size: 20779 (9.8485279083252, 0.40600800514221)
2021-06-07 21:54:10: Dupe check   (9.8486700057983, 0.00012993812561035)
2021-06-07 21:54:10: Downloading thumb http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__8ba50e04eac2a55388b5de59e95af402__double-cumshot-to-neighbours-wife-horny-pussy_0.jpg and make a copy (9.8492088317871, 0.00053906440734863)
2021-06-07 21:54:10: Identify /usr/bin//identify -verbose ../tmp/177764/tmp//107868.jpg (9.8817248344421, 0.032515048980713)
2021-06-07 21:54:10: Identify /usr/bin//identify -verbose ../tmp/177764/tmp//107868.jpg.thumb.jpg (9.932431936264, 0.050704956054688)
2021-06-07 21:54:10: Detect img size for ../tmp/177764/tmp//107868.jpg.thumb.jpg (300 x 250) (9.9834768772125, 0.051042079925537)
2021-06-07 21:54:10: Saving thumb (346760): title:Double Cumshot To Neighbours Wife Horny Pussy, desc:, function rotation_get_thumb_path (force name: ) to folder 346/ filename 760_cumshot-neighbours (9.9856359958649, 0.0021431446075439)
2021-06-07 21:54:10: Saving location: local to folder ../thumbs//346/  (9.9858520030975, 0.0002131462097168)
2021-06-07 21:54:10: save_name /760_cumshot-neighbours.jpg (9.9859519004822, 9.7036361694336E-5)
2021-06-07 21:54:10: Saving file ../tmp/177764/tmp//107868.jpg.thumb.jpg (20779 bytes) to ../thumbs//346///760_cumshot-neighbours.jpg  (9.9860458374023, 9.0837478637695E-5)
2021-06-07 21:54:10: ../thumbs//346///760_cumshot-neighbours.jpg saved (20779 bytes) (9.9862840175629, 0.00023508071899414)
2021-06-07 21:54:10: Cleanup tmp folder: ../tmp/177764 (9.9872930049896, 0.0010061264038086)
2021-06-07 21:54:10: Set gallery status to active  (9.9893448352814, 0.0020477771759033)2021-06-07 21:55:01: processed_galleries = 1, memory = 4 Mb limit 512M (0.001209020614624, 0.0012149810791016)
2021-06-07 21:55:01: Processing https://www.pornhub.com/view_video.php?viewkey=ph5dbd5e1537539 (177766) (0.014307022094727, 0.013093948364258)
2021-06-07 21:55:01: Loading gallery https://www.pornhub.com/view_video.php?viewkey=ph5dbd5e1537539 (0.017028093338013, 0.0027179718017578)
2021-06-07 21:55:05: Gallery description is empty: Update  with 'XXX Indian'  (4.7159869670868, 4.6989719867706)
2021-06-07 21:55:05: Downloading files for a custom gallery (4.7333490848541, 0.017349004745483)
2021-06-07 21:55:05: Content type: 1 (4.734393119812, 0.001032829284668)
2021-06-07 21:55:05: Creating thumb from http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__9d1d3adb8a4a1ba34638cbcacb5401e1__mrinalini-chatterjee-full-nude-uncensored-and-uncut-scene-from-grade-movie_0.jpg  (400x0) (Crop profile: 1)  (4.7358510494232, 0.001453161239624)
2021-06-07 21:55:05: Downloading img http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__9d1d3adb8a4a1ba34638cbcacb5401e1__mrinalini-chatterjee-full-nude-uncensored-and-uncut-scene-from-grade-movie_0.jpg to ../tmp/177766/tmp//110664.jpg, ref https://www.pornhub.com/view_video.php?viewkey=ph5dbd5e1537539  (4.7363269329071, 0.00046992301940918)
2021-06-07 21:55:06: Source Size: 10883 (4.8997640609741, 0.16343593597412)
2021-06-07 21:55:06: Dupe check   (4.8999230861664, 0.00014996528625488)
2021-06-07 21:55:06: Downloading thumb http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__9d1d3adb8a4a1ba34638cbcacb5401e1__mrinalini-chatterjee-full-nude-uncensored-and-uncut-scene-from-grade-movie_0.jpg and make a copy (4.9007849693298, 0.00085210800170898)
2021-06-07 21:55:06: Identify /usr/bin//identify -verbose ../tmp/177766/tmp//110664.jpg (4.9453151226044, 0.044528961181641)
2021-06-07 21:55:06: Identify /usr/bin//identify -verbose ../tmp/177766/tmp//110664.jpg.thumb.jpg (5.1267850399017, 0.18147492408752)
2021-06-07 21:55:06: Detect img size for ../tmp/177766/tmp//110664.jpg.thumb.jpg (300 x 250) (5.3064250946045, 0.17963004112244)
2021-06-07 21:55:06: Saving thumb (346761): title:Mrinalini chatterjee Full Nude Uncensored and Uncut scene from Grade Movie, desc:, function rotation_get_thumb_path (force name: ) to folder 346/ filename 761_movie (5.3120801448822, 0.0056412220001221)
2021-06-07 21:55:06: Saving location: local to folder ../thumbs//346/  (5.31378698349, 0.001701831817627)
2021-06-07 21:55:06: save_name /761_movie.jpg (5.3154430389404, 0.0016510486602783)
2021-06-07 21:55:06: Saving file ../tmp/177766/tmp//110664.jpg.thumb.jpg (10883 bytes) to ../thumbs//346///761_movie.jpg  (5.3155961036682, 0.00014495849609375)
2021-06-07 21:55:06: ../thumbs//346///761_movie.jpg saved (10883 bytes) (5.3159329891205, 0.00033211708068848)
2021-06-07 21:55:06: Cleanup tmp folder: ../tmp/177766 (5.3180451393127, 0.002108097076416)
2021-06-07 21:55:06: Set gallery status to active  (5.3257670402527, 0.0077159404754639)2021-06-07 21:55:06: processed_galleries = 2, memory = 19.3 Mb limit 512M (5.3499290943146, 0.024158000946045)
2021-06-07 21:55:06: Processing https://www.pornhub.com/view_video.php?viewkey=ph5df3941a8da65 (177767) (5.3771250247955, 0.027189970016479)
2021-06-07 21:55:06: Loading gallery https://www.pornhub.com/view_video.php?viewkey=ph5df3941a8da65 (5.4229130744934, 0.045783042907715)
2021-06-07 21:55:09: Gallery description is empty: Update  with 'XXX Indian'  (8.5019340515137, 3.0790140628815)
2021-06-07 21:55:09: Downloading files for a custom gallery (8.5114150047302, 0.0094759464263916)
2021-06-07 21:55:09: Content type: 1 (8.512020111084, 0.00059795379638672)
2021-06-07 21:55:09: Creating thumb from http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__2c7e290102c210ca53b9004ce10fe2f3__indian-neighbours-wife-cheating-husband-when-home-alone_0.jpg  (400x0) (Crop profile: 1)  (8.5128681659698, 0.00084114074707031)
2021-06-07 21:55:09: Downloading img http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__2c7e290102c210ca53b9004ce10fe2f3__indian-neighbours-wife-cheating-husband-when-home-alone_0.jpg to ../tmp/177767/tmp//848269.jpg, ref https://www.pornhub.com/view_video.php?viewkey=ph5df3941a8da65  (8.5130240917206, 0.00015115737915039)
2021-06-07 21:55:10: Source Size: 20525 (8.9406361579895, 0.42761707305908)
2021-06-07 21:55:10: Dupe check   (8.940808057785, 0.00016307830810547)
2021-06-07 21:55:10: Downloading thumb http://www.domain.pro/api/v1/pictures/thumb/300x250/data__parse__625__files__2c7e290102c210ca53b9004ce10fe2f3__indian-neighbours-wife-cheating-husband-when-home-alone_0.jpg and make a copy (8.9418880939484, 0.001072883605957)
2021-06-07 21:55:10: Identify /usr/bin//identify -verbose ../tmp/177767/tmp//848269.jpg (8.9680831432343, 0.026199102401733)
2021-06-07 21:55:10: Identify /usr/bin//identify -verbose ../tmp/177767/tmp//848269.jpg.thumb.jpg (9.0260419845581, 0.057955980300903)
2021-06-07 21:55:10: Detect img size for ../tmp/177767/tmp//848269.jpg.thumb.jpg (300 x 250) (9.0795750617981, 0.053540945053101)
2021-06-07 21:55:10: Saving thumb (346762): title:Indian Neighbours Wife Cheating Husband When Home Alone, desc:, function rotation_get_thumb_path (force name: ) to folder 346/ filename 762_indian-wife-when (9.082426071167, 0.0028209686279297)
2021-06-07 21:55:10: Saving location: local to folder ../thumbs//346/  (9.0826661586761, 0.00023603439331055)
2021-06-07 21:55:10: save_name /762_indian-wife-when.jpg (9.0828051567078, 0.00013399124145508)
2021-06-07 21:55:10: Saving file ../tmp/177767/tmp//848269.jpg.thumb.jpg (20525 bytes) to ../thumbs//346///762_indian-wife-when.jpg  (9.0829241275787, 0.00011301040649414)
2021-06-07 21:55:10: ../thumbs//346///762_indian-wife-when.jpg saved (20525 bytes) (9.0832431316376, 0.00031495094299316)
2021-06-07 21:55:10: Cleanup tmp folder: ../tmp/177767 (9.0845379829407, 0.0012900829315186)
2021-06-07 21:55:10: Set gallery status to active  (9.0876171588898, 0.00307297706604)
admin
Site Admin
Posts: 37202
Joined: Wed Sep 10, 2008 11:43 am

Re: Gallery Grabber Threads

Post by admin »

ок, скидывайте шел, я позапускаю руками
Don't forget to run script update
burn
Posts: 262
Joined: Wed Feb 01, 2017 11:27 am

Re: Gallery Grabber Threads

Post by burn »

Написал в ПМ
admin
Site Admin
Posts: 37202
Joined: Wed Sep 10, 2008 11:43 am

Re: Gallery Grabber Threads

Post by admin »

провел ап, проверим как будет сейчас
Don't forget to run script update
burn
Posts: 262
Joined: Wed Feb 01, 2017 11:27 am

Re: Gallery Grabber Threads

Post by burn »

Сейчас лучше )
53 galleries per minute
admin
Site Admin
Posts: 37202
Joined: Wed Sep 10, 2008 11:43 am

Re: Gallery Grabber Threads

Post by admin »

Отлично)
Don't forget to run script update
burn
Posts: 262
Joined: Wed Feb 01, 2017 11:27 am

Re: Gallery Grabber Threads

Post by burn »

Весь дамп отработался нормально. Примерно по 55 галер в минуту.
Спасибо.
admin
Site Admin
Posts: 37202
Joined: Wed Sep 10, 2008 11:43 am

Re: Gallery Grabber Threads

Post by admin »

отлично, NP
Don't forget to run script update
Post Reply