Здравствуйте.
наполняю туб с помощью Sets. Добавлено два Host. За 6 часов работы у первого Active - 496, To grab - 2971. У второго Active - 0, To grab - 300. Это нормальная скорость или есть проблемы?
Active To grab
Re: Active To grab
Вопрос не имеет прямого ответа
Что-то вроде "у меня груз приехал из пукнта А в пункт Б за 3 дня - это вообще нормально?"
Зависит от нагрузки сервака, сколько надо делать тумб парсить видео, как отдает удаленный хост и тп
Все это вы можете проверить в логе граббера - на каждую строку там пишет сколько времени заняло каждое действие
Что-то вроде "у меня груз приехал из пукнта А в пункт Б за 3 дня - это вообще нормально?"
Зависит от нагрузки сервака, сколько надо делать тумб парсить видео, как отдает удаленный хост и тп
Все это вы можете проверить в логе граббера - на каждую строку там пишет сколько времени заняло каждое действие
Don't forget to run script update
Re: Active To grab
Последние записи логе грабера. Нормальная скорость или должна быть выше?
2017-04-30 20:34:12: Coping thumb ... (0.028615951538086, 0.01907205581665)
2017-04-30 20:34:12: Detect img size for ../tmp/140629/tmp//535861.jpg.thumb.jpg (240 x 180) (0.028887987136841, 0.00026488304138184)
2017-04-30 20:34:12: Rot save (144558): title:LolaFick, desc:() (0.030803918838501, 0.0019159317016602)
2017-04-30 20:34:12: Saving ../thumbs/144//558_LolaFick.jpg (source ../tmp/140629/tmp//535861.jpg.thumb.jpg) (0.031126976013184, 0.00031900405883789)
2017-04-30 20:34:12: Cleanup tmp folder ../tmp/140629 (0.03158712387085, 0.00045609474182129)
2017-04-30 20:34:12: Batch done (2) (0.033102035522461, 0.0015110969543457)2017-04-30 20:35:12: Processing http://xhamster.com/movies/7613646/both ... gered.html (140630) (0.002932071685791, 0.0029380321502686)
2017-04-30 20:35:12: Gallery description is empty: Update with 'admin added' (0.0066120624542236, 0.0036768913269043)
2017-04-30 20:35:12: Content type: 1 (0.0090560913085938, 0.0024399757385254)
2017-04-30 20:35:12: Creating thumb (240x180) (PF: , Orig: x) (0.009537935256958, 0.00047802925109863)
2017-04-30 20:35:12: Downloading img http://thumb-v.xhcdn.com/t/646/240/10_7613646.jpg (../tmp/140630/tmp//38628.jpg) (0.0098121166229248, 0.00027108192443848)
2017-04-30 20:35:12: Coping thumb ... (0.025702953338623, 0.015896081924438)
2017-04-30 20:35:12: Detect img size for ../tmp/140630/tmp//38628.jpg.thumb.jpg (240 x 180) (0.0260009765625, 0.00028705596923828)
2017-04-30 20:35:12: Rot save (144559): title:Both Holes Fingered, desc:() (0.027835130691528, 0.0018320083618164)
2017-04-30 20:35:12: Saving ../thumbs/144//559_Both_Fingered.jpg (source ../tmp/140630/tmp//38628.jpg.thumb.jpg) (0.028136014938354, 0.00029706954956055)
2017-04-30 20:35:12: Cleanup tmp folder ../tmp/140630 (0.028558015823364, 0.00041794776916504)
2017-04-30 20:35:12: Batch done (2) (0.029865980148315, 0.001305103302002)2017-04-30 20:36:11: Processing http://xhamster.com/movies/7613636/fuck_russian.html (140631) (0.020793914794922, 0.020800113677979)
2017-04-30 20:36:11: Gallery description is empty: Update with 'admin added' (0.024518966674805, 0.0037219524383545)
2017-04-30 20:36:11: Content type: 1 (0.026458024978638, 0.0019330978393555)
2017-04-30 20:36:11: Creating thumb (240x180) (PF: , Orig: x) (0.026886940002441, 0.0004270076751709)
2017-04-30 20:36:11: Downloading img http://thumb-v.xhcdn.com/t/636/240/7_7613636.jpg (../tmp/140631/tmp//269669.jpg) (0.027141094207764, 0.00025081634521484)
2017-04-30 20:36:11: Coping thumb ... (0.043488025665283, 0.016348123550415)
2017-04-30 20:36:11: Detect img size for ../tmp/140631/tmp//269669.jpg.thumb.jpg (240 x 180) (0.043743133544922, 0.00024890899658203)
2017-04-30 20:36:11: Rot save (144560): title:Fuck russian, desc:() (0.045332908630371, 0.0015869140625)
2017-04-30 20:36:11: Saving ../thumbs/144//560_Fuck_russian.jpg (source ../tmp/140631/tmp//269669.jpg.thumb.jpg) (0.045602083206177, 0.00026607513427734)
2017-04-30 20:36:11: Cleanup tmp folder ../tmp/140631 (0.046030044555664, 0.00043010711669922)
2017-04-30 20:36:11: Batch done (2) (0.047441959381104, 0.0014030933380127)
2017-04-30 20:34:12: Coping thumb ... (0.028615951538086, 0.01907205581665)
2017-04-30 20:34:12: Detect img size for ../tmp/140629/tmp//535861.jpg.thumb.jpg (240 x 180) (0.028887987136841, 0.00026488304138184)
2017-04-30 20:34:12: Rot save (144558): title:LolaFick, desc:() (0.030803918838501, 0.0019159317016602)
2017-04-30 20:34:12: Saving ../thumbs/144//558_LolaFick.jpg (source ../tmp/140629/tmp//535861.jpg.thumb.jpg) (0.031126976013184, 0.00031900405883789)
2017-04-30 20:34:12: Cleanup tmp folder ../tmp/140629 (0.03158712387085, 0.00045609474182129)
2017-04-30 20:34:12: Batch done (2) (0.033102035522461, 0.0015110969543457)2017-04-30 20:35:12: Processing http://xhamster.com/movies/7613646/both ... gered.html (140630) (0.002932071685791, 0.0029380321502686)
2017-04-30 20:35:12: Gallery description is empty: Update with 'admin added' (0.0066120624542236, 0.0036768913269043)
2017-04-30 20:35:12: Content type: 1 (0.0090560913085938, 0.0024399757385254)
2017-04-30 20:35:12: Creating thumb (240x180) (PF: , Orig: x) (0.009537935256958, 0.00047802925109863)
2017-04-30 20:35:12: Downloading img http://thumb-v.xhcdn.com/t/646/240/10_7613646.jpg (../tmp/140630/tmp//38628.jpg) (0.0098121166229248, 0.00027108192443848)
2017-04-30 20:35:12: Coping thumb ... (0.025702953338623, 0.015896081924438)
2017-04-30 20:35:12: Detect img size for ../tmp/140630/tmp//38628.jpg.thumb.jpg (240 x 180) (0.0260009765625, 0.00028705596923828)
2017-04-30 20:35:12: Rot save (144559): title:Both Holes Fingered, desc:() (0.027835130691528, 0.0018320083618164)
2017-04-30 20:35:12: Saving ../thumbs/144//559_Both_Fingered.jpg (source ../tmp/140630/tmp//38628.jpg.thumb.jpg) (0.028136014938354, 0.00029706954956055)
2017-04-30 20:35:12: Cleanup tmp folder ../tmp/140630 (0.028558015823364, 0.00041794776916504)
2017-04-30 20:35:12: Batch done (2) (0.029865980148315, 0.001305103302002)2017-04-30 20:36:11: Processing http://xhamster.com/movies/7613636/fuck_russian.html (140631) (0.020793914794922, 0.020800113677979)
2017-04-30 20:36:11: Gallery description is empty: Update with 'admin added' (0.024518966674805, 0.0037219524383545)
2017-04-30 20:36:11: Content type: 1 (0.026458024978638, 0.0019330978393555)
2017-04-30 20:36:11: Creating thumb (240x180) (PF: , Orig: x) (0.026886940002441, 0.0004270076751709)
2017-04-30 20:36:11: Downloading img http://thumb-v.xhcdn.com/t/636/240/7_7613636.jpg (../tmp/140631/tmp//269669.jpg) (0.027141094207764, 0.00025081634521484)
2017-04-30 20:36:11: Coping thumb ... (0.043488025665283, 0.016348123550415)
2017-04-30 20:36:11: Detect img size for ../tmp/140631/tmp//269669.jpg.thumb.jpg (240 x 180) (0.043743133544922, 0.00024890899658203)
2017-04-30 20:36:11: Rot save (144560): title:Fuck russian, desc:() (0.045332908630371, 0.0015869140625)
2017-04-30 20:36:11: Saving ../thumbs/144//560_Fuck_russian.jpg (source ../tmp/140631/tmp//269669.jpg.thumb.jpg) (0.045602083206177, 0.00026607513427734)
2017-04-30 20:36:11: Cleanup tmp folder ../tmp/140631 (0.046030044555664, 0.00043010711669922)
2017-04-30 20:36:11: Batch done (2) (0.047441959381104, 0.0014030933380127)
Re: Active To grab
Да, обновил до последний версии.
Re: Active To grab
Сейчас лучше, Скорость стала в несколько раз больше
2017-05-01 05:16:12: Source Size 10162 (1.056627035141, 0.073318958282471)
2017-05-01 05:16:12: Make thumb ../tmp/140141/tmp//473345.jpg (1.0567860603333, 0.00014805793762207)
2017-05-01 05:16:12: Creating thumb ../tmp/140141/tmp//17186.jpg.tmp.jpg -> ../tmp/140141/tmp//17186.jpg (1.0569911003113, 0.00020313262939453)
2017-05-01 05:16:12: Identify /usr/bin//identify -verbose ../tmp/140141/tmp//17186.jpg.tmp.jpg (1.0571100711823, 0.00011515617370605)
2017-05-01 05:16:12: Size: 240x180 (../tmp/140141/tmp//17186.jpg.tmp.jpg) thumb: 240x180
thumb_ratio: 1.3333333333333 img_ratio: 1.3333333333333 to ../tmp/140141/tmp//17186.jpg (1.134761095047, 0.077653169631958)
2017-05-01 05:16:12: Face detect Cmd : /home/domains//face --input=../tmp/140141/tmp//17186.jpg.tmp.jpg --cascade=/home/domains/1.xml (1.1350350379944, 0.00026702880859375)
2017-05-01 05:16:12: Output : sh: /home/domains//face: No such file or directory
(1.163733959198, 0.028709173202515)
2017-05-01 05:16:12: IM Cmd custom : /usr/bin//convert -resize 240x180^ -gravity Center -crop 240x180+0+0 -filter Blackman -modulate 105,102,100 -enhance -sharpen 1x1 -quality 85 -strip ../tmp/140141/tmp//17186.jpg.tmp.jpg ../tmp/140141/tmp//17186.jpg (1.1640059947968, 0.00025606155395508)
2017-05-01 05:16:13: Detect img size for ../tmp/140141/tmp//17186.jpg (240 x 180) (1.2820911407471, 0.11809086799622)
2017-05-01 05:16:13: Rot save (151756): title:Magnificent Big Tits Blonde Awesome Webcam Show, desc:() (1.2842841148376, 0.0021810531616211)
2017-05-01 05:16:13: Saving ../thumbs/151//756_Blonde.jpg (source ../tmp/140141/tmp//17186.jpg) (1.2845449447632, 0.00025796890258789)
2017-05-01 05:16:13: Cleanup tmp folder ../tmp/140141 (1.284989118576, 0.00043988227844238)
2017-05-01 05:16:13: Processing http://www.drtuber.com/video/3518960/an ... es-schlong (140142) (1.2916190624237, 0.0066280364990234)
2017-05-01 05:16:13: Gallery description is empty: Update with 'admin added' (1.2927820682526, 0.0011601448059082)
2017-05-01 05:16:13: Content type: 1 (1.3098211288452, 0.017036199569702)
2017-05-01 05:16:13: Creating thumb (240x180) (Crop profile: 1) (1.3105199337006, 0.00069594383239746)
2017-05-01 05:16:13: Downloading img http://pics.drtuber.com/media/videos/tm ... 180/13.jpg (../tmp/140142/tmp//520033.jpg) (1.3106319904327, 0.00010895729064941)
2017-05-01 05:16:13: Source Size 11224 (1.3306090831757, 0.019981861114502)
2017-05-01 05:16:13: Make thumb ../tmp/140142/tmp//520033.jpg (1.3307681083679, 0.00014805793762207)
2017-05-01 05:16:13: Creating thumb ../tmp/140142/tmp//2775.jpg.tmp.jpg -> ../tmp/140142/tmp//2775.jpg (1.3309400081635, 0.00016999244689941)
2017-05-01 05:16:13: Identify /usr/bin//identify -verbose ../tmp/140142/tmp//2775.jpg.tmp.jpg (1.33109998703, 0.00015687942504883)
2017-05-01 05:16:13: Size: 240x180 (../tmp/140142/tmp//2775.jpg.tmp.jpg) thumb: 240x180
thumb_ratio: 1.3333333333333 img_ratio: 1.3333333333333 to ../tmp/140142/tmp//2775.jpg (1.4170880317688, 0.085988998413086)
2017-05-01 05:16:13: Face detect Cmd : /home/domains//face --input=../tmp/140142/tmp//2775.jpg.tmp.jpg --cascade=/home/domains/1.xml (1.4173879623413, 0.00029182434082031)
2017-05-01 05:16:13: Output : sh: /home/domains//face: No such file or directory
(1.4420199394226, 0.024642944335938)
2017-05-01 05:16:13: IM Cmd custom : /usr/bin//convert -resize 240x180^ -gravity Center -crop 240x180+0+0 -filter Blackman -modulate 105,102,100 -enhance -sharpen 1x1 -quality 85 -strip ../tmp/140142/tmp//2775.jpg.tmp.jpg ../tmp/140142/tmp//2775.jpg (1.4423079490662, 0.00026988983154297)
2017-05-01 05:16:13: Detect img size for ../tmp/140142/tmp//2775.jpg (240 x 180) (1.5044009685516, 0.062098979949951)
2017-05-01 05:16:13: Rot save (151757): title:Angelika Black Gives Footjob And Rides Schlong, desc:() (1.507187128067, 0.0027740001678467)
2017-05-01 05:16:13: Saving ../thumbs/151//757_Black_Footjob.jpg (source ../tmp/140142/tmp//2775.jpg) (1.5075089931488, 0.00031900405883789)
2017-05-01 05:16:13: Cleanup tmp folder ../tmp/140142 (1.5079441070557, 0.00043082237243652)
2017-05-01 05:16:13: Batch done (7) (1.5094101428986, 0.0014629364013672)
2017-05-01 05:16:12: Source Size 10162 (1.056627035141, 0.073318958282471)
2017-05-01 05:16:12: Make thumb ../tmp/140141/tmp//473345.jpg (1.0567860603333, 0.00014805793762207)
2017-05-01 05:16:12: Creating thumb ../tmp/140141/tmp//17186.jpg.tmp.jpg -> ../tmp/140141/tmp//17186.jpg (1.0569911003113, 0.00020313262939453)
2017-05-01 05:16:12: Identify /usr/bin//identify -verbose ../tmp/140141/tmp//17186.jpg.tmp.jpg (1.0571100711823, 0.00011515617370605)
2017-05-01 05:16:12: Size: 240x180 (../tmp/140141/tmp//17186.jpg.tmp.jpg) thumb: 240x180
thumb_ratio: 1.3333333333333 img_ratio: 1.3333333333333 to ../tmp/140141/tmp//17186.jpg (1.134761095047, 0.077653169631958)
2017-05-01 05:16:12: Face detect Cmd : /home/domains//face --input=../tmp/140141/tmp//17186.jpg.tmp.jpg --cascade=/home/domains/1.xml (1.1350350379944, 0.00026702880859375)
2017-05-01 05:16:12: Output : sh: /home/domains//face: No such file or directory
(1.163733959198, 0.028709173202515)
2017-05-01 05:16:12: IM Cmd custom : /usr/bin//convert -resize 240x180^ -gravity Center -crop 240x180+0+0 -filter Blackman -modulate 105,102,100 -enhance -sharpen 1x1 -quality 85 -strip ../tmp/140141/tmp//17186.jpg.tmp.jpg ../tmp/140141/tmp//17186.jpg (1.1640059947968, 0.00025606155395508)
2017-05-01 05:16:13: Detect img size for ../tmp/140141/tmp//17186.jpg (240 x 180) (1.2820911407471, 0.11809086799622)
2017-05-01 05:16:13: Rot save (151756): title:Magnificent Big Tits Blonde Awesome Webcam Show, desc:() (1.2842841148376, 0.0021810531616211)
2017-05-01 05:16:13: Saving ../thumbs/151//756_Blonde.jpg (source ../tmp/140141/tmp//17186.jpg) (1.2845449447632, 0.00025796890258789)
2017-05-01 05:16:13: Cleanup tmp folder ../tmp/140141 (1.284989118576, 0.00043988227844238)
2017-05-01 05:16:13: Processing http://www.drtuber.com/video/3518960/an ... es-schlong (140142) (1.2916190624237, 0.0066280364990234)
2017-05-01 05:16:13: Gallery description is empty: Update with 'admin added' (1.2927820682526, 0.0011601448059082)
2017-05-01 05:16:13: Content type: 1 (1.3098211288452, 0.017036199569702)
2017-05-01 05:16:13: Creating thumb (240x180) (Crop profile: 1) (1.3105199337006, 0.00069594383239746)
2017-05-01 05:16:13: Downloading img http://pics.drtuber.com/media/videos/tm ... 180/13.jpg (../tmp/140142/tmp//520033.jpg) (1.3106319904327, 0.00010895729064941)
2017-05-01 05:16:13: Source Size 11224 (1.3306090831757, 0.019981861114502)
2017-05-01 05:16:13: Make thumb ../tmp/140142/tmp//520033.jpg (1.3307681083679, 0.00014805793762207)
2017-05-01 05:16:13: Creating thumb ../tmp/140142/tmp//2775.jpg.tmp.jpg -> ../tmp/140142/tmp//2775.jpg (1.3309400081635, 0.00016999244689941)
2017-05-01 05:16:13: Identify /usr/bin//identify -verbose ../tmp/140142/tmp//2775.jpg.tmp.jpg (1.33109998703, 0.00015687942504883)
2017-05-01 05:16:13: Size: 240x180 (../tmp/140142/tmp//2775.jpg.tmp.jpg) thumb: 240x180
thumb_ratio: 1.3333333333333 img_ratio: 1.3333333333333 to ../tmp/140142/tmp//2775.jpg (1.4170880317688, 0.085988998413086)
2017-05-01 05:16:13: Face detect Cmd : /home/domains//face --input=../tmp/140142/tmp//2775.jpg.tmp.jpg --cascade=/home/domains/1.xml (1.4173879623413, 0.00029182434082031)
2017-05-01 05:16:13: Output : sh: /home/domains//face: No such file or directory
(1.4420199394226, 0.024642944335938)
2017-05-01 05:16:13: IM Cmd custom : /usr/bin//convert -resize 240x180^ -gravity Center -crop 240x180+0+0 -filter Blackman -modulate 105,102,100 -enhance -sharpen 1x1 -quality 85 -strip ../tmp/140142/tmp//2775.jpg.tmp.jpg ../tmp/140142/tmp//2775.jpg (1.4423079490662, 0.00026988983154297)
2017-05-01 05:16:13: Detect img size for ../tmp/140142/tmp//2775.jpg (240 x 180) (1.5044009685516, 0.062098979949951)
2017-05-01 05:16:13: Rot save (151757): title:Angelika Black Gives Footjob And Rides Schlong, desc:() (1.507187128067, 0.0027740001678467)
2017-05-01 05:16:13: Saving ../thumbs/151//757_Black_Footjob.jpg (source ../tmp/140142/tmp//2775.jpg) (1.5075089931488, 0.00031900405883789)
2017-05-01 05:16:13: Cleanup tmp folder ../tmp/140142 (1.5079441070557, 0.00043082237243652)
2017-05-01 05:16:13: Batch done (7) (1.5094101428986, 0.0014629364013672)
Re: Active To grab
Сейчас заметил, что во всех категориях есть дубли тубов, в некоторых сразу по 7 штук подряд идут. как решить эту проблему? Можно как-то удалить дубли и запретить их создавать?
Re: Active To grab
Быстрее стало ?
Grabber Speed, per minute
Сколько у вас стоит ?
Grabber Speed, per minute
Сколько у вас стоит ?
Don't forget to run script update
Re: Active To grab
Стоит 20.
С тем что было стало конечно быстрее.
С тем что было стало конечно быстрее.