На новых сиджах медленно стали грабиться галеры.
Те же дампы на старых намного быстрее. Старые на этом же сервере.
Грабится примерно по 3-4.
Выставляю Gallery Grabber Threads в 100-200 - ничего не меняется и по логам и в "Current average speed" пишет 3-4.
Сервер не нагружен: ЛА 10, 32 ядра
Крон раз в 5 мин.
Code: Select all
2021-06-05 17:32:00: Processing https://www.pornhub.com/view_video.php?viewkey=1661419764 (176403) (0.011178016662598, 0.0098631381988525)
2021-06-05 17:32:05: Set gallery status to active (5.4431891441345, 0.010816097259521)2021-06-05 17:32:05: processed_galleries = 2, memory = 26.6 Mb limit 512M (5.4459660053253, 0.0027689933776855)
2021-06-05 17:32:05: Processing https://www.pornhub.com/view_video.php?viewkey=ph5ac1f3be9232f (176404) (5.4515891075134, 0.0056159496307373)
2021-06-05 17:32:10: Set gallery status to active (9.8466761112213, 0.0017690658569336)2021-06-05 17:32:10: processed_galleries = 3, memory = 27.6 Mb limit 512M (9.8475029468536, 0.00082492828369141)
2021-06-05 17:32:10: Processing https://www.pornhub.com/view_video.php?viewkey=ph5826ceb0f2b77 (176405) (9.8497030735016, 0.0021970272064209)
2021-06-05 17:32:13: Set gallery status to active (12.871883153915, 0.0018618106842041)2021-06-05 17:37:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.00094008445739746, 0.00094509124755859)
2021-06-05 17:37:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph5bad263f27ffb (176407) (0.010069131851196, 0.0091249942779541)
2021-06-05 17:37:01: Processing https://www.pornhub.com/view_video.php?viewkey=ph566562ccc4c85 (176408) (1.4751770496368, 0.0034630298614502)
2021-06-05 17:37:06: Set gallery status to active (6.3935132026672, 0.0022909641265869)2021-06-05 17:37:06: processed_galleries = 3, memory = 26.8 Mb limit 512M (6.3946180343628, 0.0010988712310791)
2021-06-05 17:37:06: Processing https://www.pornhub.com/view_video.php?viewkey=165689273 (176409) (6.3983981609344, 0.003777027130127)
2021-06-05 17:37:10: Set gallery status to active (10.439781188965, 0.0017569065093994)2021-06-05 17:37:10: processed_galleries = 4, memory = 26.8 Mb limit 512M (10.440680027008, 0.00089597702026367)
2021-06-05 17:37:10: Processing https://www.pornhub.com/view_video.php?viewkey=ph5a533e740d918 (176410) (10.443266153336, 0.0025839805603027)
2021-06-05 17:37:12: Processing https://www.pornhub.com/view_video.php?viewkey=ph56dded9614a6c (176411) (11.791800022125, 0.0027260780334473)
2021-06-05 17:37:16: Set gallery status to active (16.002367973328, 0.0023131370544434)2021-06-05 17:37:16: processed_galleries = 6, memory = 26.8 Mb limit 512M (16.003540992737, 0.001168966293335)
2021-06-05 17:37:16: Processing https://www.pornhub.com/view_video.php?viewkey=ph59496bf0d1c5b (176412) (16.006613016129, 0.0030701160430908)
2021-06-05 17:37:19: Set gallery status to active (19.030066013336, 0.0049529075622559)2021-06-05 17:37:19: processed_galleries = 7, memory = 19.5 Mb limit 512M (19.031466007233, 0.0013968944549561)
2021-06-05 17:37:19: Processing https://www.pornhub.com/view_video.php?viewkey=ph5b6c0ff728f4a (176413) (19.03439116478, 0.0029220581054688)
2021-06-05 17:37:23: Set gallery status to active (23.03404712677, 0.002187967300415)2021-06-05 17:42:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.001011848449707, 0.0010180473327637)
2021-06-05 17:42:00: Processing https://www.pornhub.com/view_video.php?viewkey=107313928 (176415) (0.010648965835571, 0.0096371173858643)
2021-06-05 17:42:05: Set gallery status to active (5.48286485672, 0.0053369998931885)2021-06-05 17:47:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.001276969909668, 0.001284122467041)
2021-06-05 17:47:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph567c7c556bac0 (176417) (0.012966156005859, 0.011688947677612)
2021-06-05 17:47:06: Set gallery status to active (6.1631669998169, 0.0080680847167969)2021-06-05 17:52:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.0010359287261963, 0.0010418891906738)
2021-06-05 17:52:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph5a5b1e4da627c (176419) (0.010966062545776, 0.0099270343780518)
2021-06-05 17:52:06: Set gallery status to active (5.702693939209, 0.0035080909729004)2021-06-05 17:52:06: processed_galleries = 2, memory = 26.6 Mb limit 512M (5.7039148807526, 0.0012130737304688)
2021-06-05 17:52:06: Processing https://www.pornhub.com/view_video.php?viewkey=185982585 (176420) (5.7074229717255, 0.0035068988800049)
2021-06-05 17:52:10: Set gallery status to active (9.7432799339294, 0.0018038749694824)2021-06-05 17:57:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.0010690689086914, 0.0010750293731689)
2021-06-05 17:57:00: Processing https://www.pornhub.com/view_video.php?viewkey=1247813958 (176422) (0.010745048522949, 0.009674072265625)
2021-06-05 17:57:05: Set gallery status to active (5.2571499347687, 0.0061030387878418)2021-06-05 18:12:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.0010771751403809, 0.0010831356048584)
2021-06-05 18:12:00: Processing https://www.pornhub.com/view_video.php?viewkey=ph5a59179ad929b (176426) (0.010696172714233, 0.0096151828765869)
2021-06-05 18:12:06: Set gallery status to active (5.2794940471649, 0.0047140121459961)2021-06-05 18:22:00: processed_galleries = 1, memory = 4 Mb limit 512M (0.0011310577392578, 0.0011389255523682)
2021-06-05 18:22:00: Processing https://www.pornhub.com/view_video.php?viewkey=748228653 (176429) (0.011747121810913, 0.010611057281494)
2021-06-05 18:22:05: Set gallery status to active (5.422110080719, 0.0063021183013916)2021-06-05 18:22:05: processed_galleries = 2, memory = 26.9 Mb limit 512M (5.4259519577026, 0.0038361549377441)