При импорте через ипмпорт сет

Post Reply
darkover
Posts: 1574
Joined: Sat Apr 13, 2013 8:59 pm

При импорте через ипмпорт сет

Post by darkover »

Привет! Вопрос такой
При импорте через импорт сетс, происходит проверка по url и source url на предмет дублей ?
У меня интересная ситуация тут вылезла
Одна и тажа галера залезла 2 раза

Code: Select all

https://jbtube.co/1576976/korean-milf-with-giant-tits-mia-rider-gets-roughly-fucked-up-her-butt.html
korean-milf-with-giant-tits-mia-rider-gets-roughly
 Gallery id : 851536 (2017-12-26) 
 
https://jbtube.co/1576976/korean-milf-with-giant-tits-mia-rider-gets-roughly-fucked-up-her-butt.html
korean-milf-with-giant-tits-mia-rider-gets-roughly
 Gallery id : 851537 (2017-12-26)
Check DB for Dupes - Yes
Top
darkover
Posts: 1574
Joined: Sat Apr 13, 2013 8:59 pm

Re: При импорте через ипмпорт сет

Post by darkover »

Еще интересный момент #2

wc -l ./voyeurhit_video.csv
151802 ./voyeurhit_video.csv
Импорт сет загрузился, и опять пошел по новой загружаться, никаких записей нет, что process big file once pause после 100%, опять по новой пошел грузиться, судя по % которые показывает импорт сет
Top
admin
Site Admin
Posts: 38004
Joined: Wed Sep 10, 2008 11:43 am

Re: При импорте через ипмпорт сет

Post by admin »

darkover wrote: Sun Nov 07, 2021 7:51 pm Еще интересный момент #2

wc -l ./voyeurhit_video.csv
151802 ./voyeurhit_video.csv
Импорт сет загрузился, и опять пошел по новой загружаться, никаких записей нет, что process big file once pause после 100%, опять по новой пошел грузиться, судя по % которые показывает импорт сет
по этому описанию не ясно что именно не так происходит
скиньте плз админку и лучше сразу шел
Don't forget to run script update
Top
darkover
Posts: 1574
Joined: Sat Apr 13, 2013 8:59 pm

Re: При импорте через ипмпорт сет

Post by darkover »

admin wrote: Mon Nov 08, 2021 6:11 am
darkover wrote: Sun Nov 07, 2021 7:51 pm Еще интересный момент #2

wc -l ./voyeurhit_video.csv
151802 ./voyeurhit_video.csv
Импорт сет загрузился, и опять пошел по новой загружаться, никаких записей нет, что process big file once pause после 100%, опять по новой пошел грузиться, судя по % которые показывает импорт сет
по этому описанию не ясно что именно не так происходит
скиньте плз админку и лучше сразу шел
Я базу эту скидывал же, это в ней траблы какие-то. Я смотрю, большие импорт сеты доходят до 100 и опа, пошли заного, в логе ротации, если грепнуть то видно вот такое

Code: Select all

2021-11-08 20:40:50: ImportSet: Lines processed: 480, Added: 333, Dupes: 143 Skip: 147 (48.795555114746, 0.0073919296264648)
2021-11-08 20:40:51: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (110.79776787758, 0.0058021545410156)
2021-11-08 20:40:51: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (169.99026608467, 0.0067799091339111)
2021-11-08 20:40:52: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (112.09355592728, 0.0047030448913574)
2021-11-08 20:40:52: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (171.26713705063, 0.011567831039429)
2021-11-08 20:42:49: ImportSet: Lines processed: 500, Added: 500, Dupes:  Skip:  (167.78173303604, 0.0032889842987061)
2021-11-08 20:43:03: ImportSet: Lines processed: 490, Added: 347, Dupes: 133 Skip: 143 (122.71974396706, 0.0040979385375977)
2021-11-08 20:43:16: ImportSet: Lines processed: 495, Added: 423, Dupes: 63 Skip: 72 (74.570912837982, 0.007728099822998)
2021-11-08 20:43:25: ImportSet: Lines processed: 500, Added: 181, Dupes: 310 Skip: 319 (83.060336828232, 0.0040640830993652)
2021-11-08 20:43:25: ImportSet: Lines processed: 500, Added: 451, Dupes:  Skip: 49 (144.20425701141, 0.0033729076385498)
2021-11-08 20:43:44: ImportSet: Lines processed: 494, Added: 433, Dupes: 57 Skip: 61 (43.432136058807, 0.0022480487823486)
2021-11-08 20:43:45: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (44.389288902283, 0.0023560523986816)
2021-11-08 20:44:18: ImportSet: Lines processed: 500, Added: 401, Dupes: 41 Skip: 99 (257.14766097069, 0.0032451152801514)
2021-11-08 20:44:18: ImportSet: Lines processed: 500, Added: 99, Dupes: 277 Skip: 401 (198.0488910675, 0.0088841915130615)
2021-11-08 20:44:18: ImportSet: Lines processed: 500, Added: 112, Dupes: 281 Skip: 388 (136.91894197464, 0.0068871974945068)
2021-11-08 20:44:20: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (258.33232998848, 0.0037200450897217)
2021-11-08 20:44:20: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (138.14088201523, 0.003864049911499)
2021-11-08 20:44:20: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (199.28931713104, 0.010480880737305)
2021-11-08 20:44:20: ImportSet: Lines processed: , Added: , Dupes:  Skip:  (199.69575309753, 0.39588499069214)
2021-11-08 20:44:20: ImportSet: Lines processed: , Added: , Dupes:  Skip:  (138.58902287483, 0.44747996330261)
2021-11-08 20:44:21: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (260.16461896896, 0.0063700675964355)
2021-11-08 20:44:21: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (139.9386048317, 0.0042519569396973)
2021-11-08 20:44:21: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (201.0963909626, 0.0051858425140381)
2021-11-08 20:44:23: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (261.45159316063, 0.00364089012146)
2021-11-08 20:44:23: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (141.2611758709, 0.0044400691986084)
2021-11-08 20:44:23: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (202.46990299225, 0.0044479370117188)
2021-11-08 20:44:24: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (262.53008103371, 0.004472017288208)
2021-11-08 20:44:24: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (142.49999690056, 0.0046169757843018)
2021-11-08 20:44:24: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (203.73004508018, 0.0050630569458008)
2021-11-08 20:44:25: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (263.62963604927, 0.0051100254058838)
2021-11-08 20:44:25: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (143.84553790092, 0.004817008972168)
2021-11-08 20:44:25: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (205.04057407379, 0.0056731700897217)
2021-11-08 20:44:51: ImportSet: Lines processed: 495, Added: 359, Dupes: 132 Skip: 136 (50.977075099945, 0.0073370933532715)
2021-11-08 20:45:12: ImportSet: Lines processed: 500, Added: 500, Dupes:  Skip:  (72.358481168747, 0.007483959197998)
2021-11-08 20:45:26: ImportSet: Lines processed: 500, Added: 500, Dupes:  Skip:  (145.19891405106, 0.0081830024719238)
2021-11-08 20:45:27: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (146.55070996284, 0.0032360553741455)
2021-11-08 20:45:28: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (147.70354294777, 0.0042598247528076)
2021-11-08 20:45:29: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (148.76400208473, 0.0042660236358643)
2021-11-08 20:45:31: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (150.03326511383, 0.004429817199707)
2021-11-08 20:45:32: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (151.24317908287, 0.0046157836914062)
2021-11-08 20:45:33: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (152.65903091431, 0.0043618679046631)
2021-11-08 20:46:01: ImportSet: Lines processed: 497, Added: 387, Dupes: 107 Skip: 110 (60.046064138412, 0.0080859661102295)
2021-11-08 20:46:21: ImportSet: Lines processed: 500, Added: 500, Dupes:  Skip:  (80.597321033478, 0.0080199241638184)
2021-11-08 20:46:36: ImportSet: Lines processed: 481, Added: 403, Dupes: 73 Skip: 78 (36.476821899414, 0.0025651454925537)
2021-11-08 20:46:37: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (37.340004920959, 0.0023720264434814)
2021-11-08 20:46:45: ImportSet: Lines processed: 500, Added: 82, Dupes: 394 Skip: 418 (104.53840994835, 0.0066919326782227)
2021-11-08 20:46:45: ImportSet: Lines processed: 500, Added: 448, Dupes: 14 Skip: 52 (165.66895198822, 0.012894868850708)
2021-11-08 20:46:47: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (166.7639901638, 0.0041530132293701)
2021-11-08 20:46:47: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (105.6636660099, 0.0067651271820068)
2021-11-08 20:46:48: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (106.64897012711, 0.010190963745117)
2021-11-08 20:46:48: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (167.91363811493, 0.0071091651916504)
2021-11-08 20:46:49: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (107.83493494987, 0.0042879581451416)
2021-11-08 20:46:49: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (169.0659930706, 0.0041429996490479)
2021-11-08 20:46:50: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (109.12033700943, 0.0042469501495361)
2021-11-08 20:46:50: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (170.29496717453, 0.0051658153533936)
2021-11-08 20:46:51: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (171.52474808693, 0.0049209594726562)
2021-11-08 20:46:51: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (110.45181202888, 0.0055508613586426)
2021-11-08 20:46:52: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (111.55911517143, 0.0087888240814209)
2021-11-08 20:46:53: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (172.7666079998, 0.0057170391082764)
2021-11-08 20:47:32: ImportSet: Lines processed: 493, Added: 348, Dupes: 141 Skip: 145 (31.098923921585, 0.0041289329528809)
2021-11-08 20:47:55: ImportSet: Lines processed: 500, Added: 500, Dupes:  Skip:  (53.664930105209, 0.0020208358764648)
2021-11-08 20:48:09: ImportSet: Lines processed: 500, Added: 500, Dupes:  Skip:  (128.9403719902, 0.0028669834136963)
2021-11-08 20:48:10: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (130.08428287506, 0.003277063369751)
2021-11-08 20:48:11: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (131.30017280579, 0.004561185836792)
2021-11-08 20:48:13: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (132.53272485733, 0.0043699741363525)
2021-11-08 20:48:14: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (133.71209788322, 0.004525899887085)
2021-11-08 20:48:15: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (134.83667898178, 0.0043601989746094)
2021-11-08 20:48:16: ImportSet: Lines processed: 500, Added: , Dupes: 500 Skip: 500 (136.15613698959, 0.004647970199585)
Top
admin
Site Admin
Posts: 38004
Joined: Wed Sep 10, 2008 11:43 am

Re: При импорте через ипмпорт сет

Post by admin »

лучше админку скиньте с шелом а то разворачивать базу в 15М каждый раз когда какие-то вопросы - долгий процесс


и доходит до 100 чего? это проценты или что ?
Don't forget to run script update
Top
darkover
Posts: 1574
Joined: Sat Apr 13, 2013 8:59 pm

Re: При импорте через ипмпорт сет

Post by darkover »

admin wrote: Tue Nov 09, 2021 6:05 am лучше админку скиньте с шелом а то разворачивать базу в 15М каждый раз когда какие-то вопросы - долгий процесс


и доходит до 100 чего? это проценты или что ?
Ога, вот этот уже один раз был 100% ) Потом заного, и вот уже 94% снова
http://my.jetscreenshot.com/demo/20211109-xixm-17kb.jpg
Top
admin
Site Admin
Posts: 38004
Joined: Wed Sep 10, 2008 11:43 am

Re: При импорте через ипмпорт сет

Post by admin »

да, ответ тот же что и выше)
Don't forget to run script update
Top
Post Reply

Return to “Поддержка SmartCJ”

Your Domains

You'll see your domains here once you log in