Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................................X..........X....X.........] duration: 1.853249343s FAIL (2025-08-07 18:16:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.853276464s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X............................XXX.......X..X.................] duration: 6.053319595s FAIL (2025-08-07 18:15:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 6.053387785s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................................X..........X....X........] duration: 1.93124922s FAIL (2025-08-07 18:15:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.931274367s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X............................XXX.......X..X................] duration: 5.811070251s FAIL (2025-08-07 18:14:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.811096111s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................................X..........X....X.......] duration: 1.954306466s FAIL (2025-08-07 18:14:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.95437161s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X............................XXX.......X..X...............] duration: 5.756395253s FAIL (2025-08-07 18:14:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.756419398s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X............................XXX.......X..X..............] duration: 5.965376093s FAIL (2025-08-07 18:14:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.965403254s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X..........X....X......] duration: 1.83407135s FAIL (2025-08-07 18:13:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.8340919s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................................X..........X....X.....] duration: 1.844557476s FAIL (2025-08-07 18:13:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.844583204s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X............................XXX.......X..X.............] duration: 5.964031348s FAIL (2025-08-07 18:13:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.964056827s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X............................XXX.......X..X............] duration: 5.993355205s FAIL (2025-08-07 18:12:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.993373018s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X..........X....X....] duration: 1.932907513s FAIL (2025-08-07 18:12:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.932952769s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X............................XXX.......X..X...........] duration: 6.005223021s FAIL (2025-08-07 18:12:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 6.0052487s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................................X..........X....X...] duration: 2.210225381s FAIL (2025-08-07 18:11:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 2.210247815s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X............................XXX.......X..X..........] duration: 5.865736404s FAIL (2025-08-07 18:11:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.865759156s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X..........X....X..] duration: 1.867098902s FAIL (2025-08-07 18:11:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.867127186s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X............................XXX.......X..X.........] duration: 5.892754638s FAIL (2025-08-07 18:10:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.892782731s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X..........X....X.] duration: 1.840901258s FAIL (2025-08-07 18:10:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.840969889s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X............................XXX.......X..X........] duration: 5.828637773s FAIL (2025-08-07 18:09:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.82866754s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X..........X....X] duration: 1.3919537s FAIL (2025-08-07 18:09:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Aug 7 18:06:58.759713181 (8h26m58.349058158s) [upload=8824]: Starting upload Aug 7 18:06:58.759765702 (8h26m58.349110689s) [upload=8824]: Done starting upload Aug 7 18:06:58.759814064 (8h26m58.349159052s) [upload=8824]: Uploading segments... Aug 7 18:06:58.759845304 (8h26m58.349190291s) [upload=8824]: Got next segment. Inline: true Aug 7 18:06:58.759849382 (8h26m58.349194369s) [upload=8824]: Waiting for error group managing segments... Aug 7 18:06:58.759883998 (8h26m58.349228985s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:58.759888256 (8h26m58.349233243s) [upload=8824]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:58.759892514 (8h26m58.349237501s) [upload=8824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:59.029430588 (8h26m58.618775575s) [upload=8824]: Done uploading segments... Aug 7 18:06:59.029447962 (8h26m58.618792949s) [upload=8824]: Upload finished. err: <nil> Aug 7 18:07:21.849459220 (8h27m21.438804207s) [upload=8832]: Starting upload Aug 7 18:07:21.849502753 (8h27m21.43884774s) [upload=8832]: Done starting upload Aug 7 18:07:21.849555023 (8h27m21.43890001s) [upload=8832]: Uploading segments... Aug 7 18:07:21.849584629 (8h27m21.438929606s) [upload=8832]: Got next segment. Inline: true Aug 7 18:07:21.849588207 (8h27m21.438933194s) [upload=8832]: Waiting for error group managing segments... Aug 7 18:07:21.849642782 (8h27m21.438987769s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:07:21.849646969 (8h27m21.438991946s) [upload=8832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:07:21.849649234 (8h27m21.438994221s) [upload=8832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:07:22.125155216 (8h27m21.714500193s) [upload=8832]: Done uploading segments... Aug 7 18:07:22.125168983 (8h27m21.71451397s) [upload=8832]: Upload finished. err: <nil> Aug 7 18:08:04.258700436 (8h28m3.848045423s) [upload=8844]: Starting upload Aug 7 18:08:04.258764518 (8h28m3.848109505s) [upload=8844]: Done starting upload Aug 7 18:08:04.258817950 (8h28m3.848162937s) [upload=8844]: Uploading segments... Aug 7 18:08:04.258851825 (8h28m3.848196812s) [upload=8844]: Got next segment. Inline: true Aug 7 18:08:04.258856134 (8h28m3.848201121s) [upload=8844]: Waiting for error group managing segments... Aug 7 18:08:04.258889087 (8h28m3.848234074s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:08:04.258893615 (8h28m3.848238602s) [upload=8844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:08:04.258896881 (8h28m3.848241868s) [upload=8844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:08:04.739572923 (8h28m4.32891791s) [upload=8844]: Done uploading segments... Aug 7 18:08:04.739591489 (8h28m4.328936476s) [upload=8844]: Upload finished. err: <nil> Aug 7 18:09:08.962290352 (8h29m8.551635339s) [upload=8862]: Starting upload Aug 7 18:09:08.962325299 (8h29m8.551670286s) [upload=8862]: Done starting upload Aug 7 18:09:08.962367259 (8h29m8.551712246s) [upload=8862]: Uploading segments... Aug 7 18:09:08.962397838 (8h29m8.551742825s) [upload=8862]: Got next segment. Inline: true Aug 7 18:09:08.962401585 (8h29m8.551746572s) [upload=8862]: Waiting for error group managing segments... Aug 7 18:09:08.962444857 (8h29m8.551789844s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:08.962449055 (8h29m8.551794042s) [upload=8862]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:08.962451691 (8h29m8.551796678s) [upload=8862]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:09.256530018 (8h29m8.845875005s) [upload=8862]: Done uploading segments... Aug 7 18:09:09.256598799 (8h29m8.845943796s) [upload=8862]: Upload finished. err: <nil> Aug 7 18:09:53.903185086 (8h29m53.492530073s) [upload=8874]: Starting upload Aug 7 18:09:53.903229430 (8h29m53.492574417s) [upload=8874]: Done starting upload Aug 7 18:09:53.903289726 (8h29m53.492634714s) [upload=8874]: Uploading segments... Aug 7 18:09:53.903323862 (8h29m53.492668849s) [upload=8874]: Got next segment. Inline: true Aug 7 18:09:53.903327488 (8h29m53.492672465s) [upload=8874]: Waiting for error group managing segments... Aug 7 18:09:53.903359990 (8h29m53.492704977s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:09:53.903363988 (8h29m53.492708975s) [upload=8874]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:09:53.903366563 (8h29m53.49271154s) [upload=8874]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:09:54.205270072 (8h29m53.794615059s) [upload=8874]: Done uploading segments... Aug 7 18:09:54.205287746 (8h29m53.794632733s) [upload=8874]: Upload finished. err: <nil> Duration: 1.391982385s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X............................XXX.......X..X.......] duration: 5.869346089s FAIL (2025-08-07 18:09:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.86937255s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X..........X....] duration: 2.005565688s FAIL (2025-08-07 18:09:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f931041cc3fd4f718673115b0ced3b75c502d0de3062da613e9263d5f1540ce9 != before d9d564b93c13f9f4e42038997ef8441abcb4d9316ab3b7b7504a89b217bb5e34 ======LOGS====== : pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:36.950552141 (8h19m36.539897128s) [upload=8693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:37.231511518 (8h19m36.820856505s) [upload=8693]: Done uploading segments... Aug 7 17:59:37.231531727 (8h19m36.820876714s) [upload=8693]: Upload finished. err: <nil> Aug 7 17:59:56.825653287 (8h19m56.414998274s) [upload=8699]: Starting upload Aug 7 17:59:56.825697972 (8h19m56.415042959s) [upload=8699]: Done starting upload Aug 7 17:59:56.825753909 (8h19m56.415098896s) [upload=8699]: Uploading segments... Aug 7 17:59:56.825786031 (8h19m56.415131018s) [upload=8699]: Got next segment. Inline: true Aug 7 17:59:56.825789608 (8h19m56.415134595s) [upload=8699]: Waiting for error group managing segments... Aug 7 17:59:56.825853870 (8h19m56.415198858s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:56.825858169 (8h19m56.415203146s) [upload=8699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:56.825860583 (8h19m56.41520557s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:57.105776123 (8h19m56.69512111s) [upload=8699]: Done uploading segments... Aug 7 17:59:57.105818965 (8h19m56.695163952s) [upload=8699]: Upload finished. err: <nil> Aug 7 18:01:03.618992056 (8h21m3.208337044s) [upload=8716]: Starting upload Aug 7 18:01:03.619055839 (8h21m3.208400826s) [upload=8716]: Done starting upload Aug 7 18:01:03.619137906 (8h21m3.208482893s) [upload=8716]: Uploading segments... Aug 7 18:01:03.619245863 (8h21m3.20859085s) [upload=8716]: Got next segment. Inline: true Aug 7 18:01:03.619250782 (8h21m3.208595769s) [upload=8716]: Waiting for error group managing segments... Aug 7 18:01:03.619294195 (8h21m3.208639182s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:03.619299145 (8h21m3.208644132s) [upload=8716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:01:03.619302010 (8h21m3.208646997s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:03.916050629 (8h21m3.505395616s) [upload=8716]: Done uploading segments... Aug 7 18:01:03.916069315 (8h21m3.505414302s) [upload=8716]: Upload finished. err: <nil> Aug 7 18:02:01.464319909 (8h22m1.053664906s) [upload=8733]: Starting upload Aug 7 18:02:01.464389803 (8h22m1.05373479s) [upload=8733]: Done starting upload Aug 7 18:02:01.464453204 (8h22m1.053798191s) [upload=8733]: Uploading segments... Aug 7 18:02:01.464502348 (8h22m1.053847335s) [upload=8733]: Got next segment. Inline: true Aug 7 18:02:01.464506175 (8h22m1.053851163s) [upload=8733]: Waiting for error group managing segments... Aug 7 18:02:01.464543306 (8h22m1.053888293s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:01.464547554 (8h22m1.053892541s) [upload=8733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:01.464550390 (8h22m1.053895378s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:01.743231809 (8h22m1.332576796s) [upload=8733]: Done uploading segments... Aug 7 18:02:01.743251787 (8h22m1.332596774s) [upload=8733]: Upload finished. err: <nil> Aug 7 18:02:42.581972343 (8h22m42.17131732s) [upload=8748]: Starting upload Aug 7 18:02:42.582022770 (8h22m42.171367747s) [upload=8748]: Done starting upload Aug 7 18:02:42.582068957 (8h22m42.171413945s) [upload=8748]: Uploading segments... Aug 7 18:02:42.582097883 (8h22m42.17144287s) [upload=8748]: Got next segment. Inline: true Aug 7 18:02:42.582102943 (8h22m42.17144793s) [upload=8748]: Waiting for error group managing segments... Aug 7 18:02:42.582136157 (8h22m42.171481134s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:42.582140294 (8h22m42.171485281s) [upload=8748]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:42.582142659 (8h22m42.171487646s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:42.866402070 (8h22m42.455747057s) [upload=8748]: Done uploading segments... Aug 7 18:02:42.866423541 (8h22m42.455768529s) [upload=8748]: Upload finished. err: <nil> Aug 7 18:02:58.324030621 (8h22m57.913375608s) [upload=8751]: Starting upload Aug 7 18:02:58.324074084 (8h22m57.913419071s) [upload=8751]: Done starting upload Aug 7 18:02:58.324126124 (8h22m57.913471101s) [upload=8751]: Uploading segments... Aug 7 18:02:58.324157975 (8h22m57.913502952s) [upload=8751]: Got next segment. Inline: true Aug 7 18:02:58.324164246 (8h22m57.913509233s) [upload=8751]: Waiting for error group managing segments... Aug 7 18:02:58.324196638 (8h22m57.913541625s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:58.324200235 (8h22m57.913545222s) [upload=8751]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Duration: 2.005596107s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X............................XXX.......X..X......] duration: 6.026199876s FAIL (2025-08-07 18:08:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 6.026229714s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [............X............................XXX.......X..X.....] duration: 5.782470521s FAIL (2025-08-07 18:08:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.782499618s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X..........X...] duration: 2.169552513s FAIL (2025-08-07 18:08:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f931041cc3fd4f718673115b0ced3b75c502d0de3062da613e9263d5f1540ce9 != before d9d564b93c13f9f4e42038997ef8441abcb4d9316ab3b7b7504a89b217bb5e34 ======LOGS====== : pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:36.950552141 (8h19m36.539897128s) [upload=8693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:37.231511518 (8h19m36.820856505s) [upload=8693]: Done uploading segments... Aug 7 17:59:37.231531727 (8h19m36.820876714s) [upload=8693]: Upload finished. err: <nil> Aug 7 17:59:56.825653287 (8h19m56.414998274s) [upload=8699]: Starting upload Aug 7 17:59:56.825697972 (8h19m56.415042959s) [upload=8699]: Done starting upload Aug 7 17:59:56.825753909 (8h19m56.415098896s) [upload=8699]: Uploading segments... Aug 7 17:59:56.825786031 (8h19m56.415131018s) [upload=8699]: Got next segment. Inline: true Aug 7 17:59:56.825789608 (8h19m56.415134595s) [upload=8699]: Waiting for error group managing segments... Aug 7 17:59:56.825853870 (8h19m56.415198858s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:56.825858169 (8h19m56.415203146s) [upload=8699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:56.825860583 (8h19m56.41520557s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:57.105776123 (8h19m56.69512111s) [upload=8699]: Done uploading segments... Aug 7 17:59:57.105818965 (8h19m56.695163952s) [upload=8699]: Upload finished. err: <nil> Aug 7 18:01:03.618992056 (8h21m3.208337044s) [upload=8716]: Starting upload Aug 7 18:01:03.619055839 (8h21m3.208400826s) [upload=8716]: Done starting upload Aug 7 18:01:03.619137906 (8h21m3.208482893s) [upload=8716]: Uploading segments... Aug 7 18:01:03.619245863 (8h21m3.20859085s) [upload=8716]: Got next segment. Inline: true Aug 7 18:01:03.619250782 (8h21m3.208595769s) [upload=8716]: Waiting for error group managing segments... Aug 7 18:01:03.619294195 (8h21m3.208639182s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:03.619299145 (8h21m3.208644132s) [upload=8716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:01:03.619302010 (8h21m3.208646997s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:03.916050629 (8h21m3.505395616s) [upload=8716]: Done uploading segments... Aug 7 18:01:03.916069315 (8h21m3.505414302s) [upload=8716]: Upload finished. err: <nil> Aug 7 18:02:01.464319909 (8h22m1.053664906s) [upload=8733]: Starting upload Aug 7 18:02:01.464389803 (8h22m1.05373479s) [upload=8733]: Done starting upload Aug 7 18:02:01.464453204 (8h22m1.053798191s) [upload=8733]: Uploading segments... Aug 7 18:02:01.464502348 (8h22m1.053847335s) [upload=8733]: Got next segment. Inline: true Aug 7 18:02:01.464506175 (8h22m1.053851163s) [upload=8733]: Waiting for error group managing segments... Aug 7 18:02:01.464543306 (8h22m1.053888293s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:01.464547554 (8h22m1.053892541s) [upload=8733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:01.464550390 (8h22m1.053895378s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:01.743231809 (8h22m1.332576796s) [upload=8733]: Done uploading segments... Aug 7 18:02:01.743251787 (8h22m1.332596774s) [upload=8733]: Upload finished. err: <nil> Aug 7 18:02:42.581972343 (8h22m42.17131732s) [upload=8748]: Starting upload Aug 7 18:02:42.582022770 (8h22m42.171367747s) [upload=8748]: Done starting upload Aug 7 18:02:42.582068957 (8h22m42.171413945s) [upload=8748]: Uploading segments... Aug 7 18:02:42.582097883 (8h22m42.17144287s) [upload=8748]: Got next segment. Inline: true Aug 7 18:02:42.582102943 (8h22m42.17144793s) [upload=8748]: Waiting for error group managing segments... Aug 7 18:02:42.582136157 (8h22m42.171481134s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:42.582140294 (8h22m42.171485281s) [upload=8748]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:42.582142659 (8h22m42.171487646s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:42.866402070 (8h22m42.455747057s) [upload=8748]: Done uploading segments... Aug 7 18:02:42.866423541 (8h22m42.455768529s) [upload=8748]: Upload finished. err: <nil> Aug 7 18:02:58.324030621 (8h22m57.913375608s) [upload=8751]: Starting upload Aug 7 18:02:58.324074084 (8h22m57.913419071s) [upload=8751]: Done starting upload Aug 7 18:02:58.324126124 (8h22m57.913471101s) [upload=8751]: Uploading segments... Aug 7 18:02:58.324157975 (8h22m57.913502952s) [upload=8751]: Got next segment. Inline: true Aug 7 18:02:58.324164246 (8h22m57.913509233s) [upload=8751]: Waiting for error group managing segments... Aug 7 18:02:58.324196638 (8h22m57.913541625s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:58.324200235 (8h22m57.913545222s) [upload=8751]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Duration: 2.169579624s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............X............................XXX.......X..X....] duration: 6.027304004s FAIL (2025-08-07 18:07:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 6.027321606s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X..........X..] duration: 1.842524889s FAIL (2025-08-07 18:07:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f931041cc3fd4f718673115b0ced3b75c502d0de3062da613e9263d5f1540ce9 != before d9d564b93c13f9f4e42038997ef8441abcb4d9316ab3b7b7504a89b217bb5e34 ======LOGS====== : pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:36.950552141 (8h19m36.539897128s) [upload=8693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:37.231511518 (8h19m36.820856505s) [upload=8693]: Done uploading segments... Aug 7 17:59:37.231531727 (8h19m36.820876714s) [upload=8693]: Upload finished. err: <nil> Aug 7 17:59:56.825653287 (8h19m56.414998274s) [upload=8699]: Starting upload Aug 7 17:59:56.825697972 (8h19m56.415042959s) [upload=8699]: Done starting upload Aug 7 17:59:56.825753909 (8h19m56.415098896s) [upload=8699]: Uploading segments... Aug 7 17:59:56.825786031 (8h19m56.415131018s) [upload=8699]: Got next segment. Inline: true Aug 7 17:59:56.825789608 (8h19m56.415134595s) [upload=8699]: Waiting for error group managing segments... Aug 7 17:59:56.825853870 (8h19m56.415198858s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:56.825858169 (8h19m56.415203146s) [upload=8699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:56.825860583 (8h19m56.41520557s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:57.105776123 (8h19m56.69512111s) [upload=8699]: Done uploading segments... Aug 7 17:59:57.105818965 (8h19m56.695163952s) [upload=8699]: Upload finished. err: <nil> Aug 7 18:01:03.618992056 (8h21m3.208337044s) [upload=8716]: Starting upload Aug 7 18:01:03.619055839 (8h21m3.208400826s) [upload=8716]: Done starting upload Aug 7 18:01:03.619137906 (8h21m3.208482893s) [upload=8716]: Uploading segments... Aug 7 18:01:03.619245863 (8h21m3.20859085s) [upload=8716]: Got next segment. Inline: true Aug 7 18:01:03.619250782 (8h21m3.208595769s) [upload=8716]: Waiting for error group managing segments... Aug 7 18:01:03.619294195 (8h21m3.208639182s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:03.619299145 (8h21m3.208644132s) [upload=8716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:01:03.619302010 (8h21m3.208646997s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:03.916050629 (8h21m3.505395616s) [upload=8716]: Done uploading segments... Aug 7 18:01:03.916069315 (8h21m3.505414302s) [upload=8716]: Upload finished. err: <nil> Aug 7 18:02:01.464319909 (8h22m1.053664906s) [upload=8733]: Starting upload Aug 7 18:02:01.464389803 (8h22m1.05373479s) [upload=8733]: Done starting upload Aug 7 18:02:01.464453204 (8h22m1.053798191s) [upload=8733]: Uploading segments... Aug 7 18:02:01.464502348 (8h22m1.053847335s) [upload=8733]: Got next segment. Inline: true Aug 7 18:02:01.464506175 (8h22m1.053851163s) [upload=8733]: Waiting for error group managing segments... Aug 7 18:02:01.464543306 (8h22m1.053888293s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:01.464547554 (8h22m1.053892541s) [upload=8733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:01.464550390 (8h22m1.053895378s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:01.743231809 (8h22m1.332576796s) [upload=8733]: Done uploading segments... Aug 7 18:02:01.743251787 (8h22m1.332596774s) [upload=8733]: Upload finished. err: <nil> Aug 7 18:02:42.581972343 (8h22m42.17131732s) [upload=8748]: Starting upload Aug 7 18:02:42.582022770 (8h22m42.171367747s) [upload=8748]: Done starting upload Aug 7 18:02:42.582068957 (8h22m42.171413945s) [upload=8748]: Uploading segments... Aug 7 18:02:42.582097883 (8h22m42.17144287s) [upload=8748]: Got next segment. Inline: true Aug 7 18:02:42.582102943 (8h22m42.17144793s) [upload=8748]: Waiting for error group managing segments... Aug 7 18:02:42.582136157 (8h22m42.171481134s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:42.582140294 (8h22m42.171485281s) [upload=8748]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:42.582142659 (8h22m42.171487646s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:42.866402070 (8h22m42.455747057s) [upload=8748]: Done uploading segments... Aug 7 18:02:42.866423541 (8h22m42.455768529s) [upload=8748]: Upload finished. err: <nil> Aug 7 18:02:58.324030621 (8h22m57.913375608s) [upload=8751]: Starting upload Aug 7 18:02:58.324074084 (8h22m57.913419071s) [upload=8751]: Done starting upload Aug 7 18:02:58.324126124 (8h22m57.913471101s) [upload=8751]: Uploading segments... Aug 7 18:02:58.324157975 (8h22m57.913502952s) [upload=8751]: Got next segment. Inline: true Aug 7 18:02:58.324164246 (8h22m57.913509233s) [upload=8751]: Waiting for error group managing segments... Aug 7 18:02:58.324196638 (8h22m57.913541625s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:58.324200235 (8h22m57.913545222s) [upload=8751]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Duration: 1.842556307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X..........X.] duration: 1.867523427s FAIL (2025-08-07 18:06:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f931041cc3fd4f718673115b0ced3b75c502d0de3062da613e9263d5f1540ce9 != before d9d564b93c13f9f4e42038997ef8441abcb4d9316ab3b7b7504a89b217bb5e34 ======LOGS====== : pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:36.950552141 (8h19m36.539897128s) [upload=8693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:37.231511518 (8h19m36.820856505s) [upload=8693]: Done uploading segments... Aug 7 17:59:37.231531727 (8h19m36.820876714s) [upload=8693]: Upload finished. err: <nil> Aug 7 17:59:56.825653287 (8h19m56.414998274s) [upload=8699]: Starting upload Aug 7 17:59:56.825697972 (8h19m56.415042959s) [upload=8699]: Done starting upload Aug 7 17:59:56.825753909 (8h19m56.415098896s) [upload=8699]: Uploading segments... Aug 7 17:59:56.825786031 (8h19m56.415131018s) [upload=8699]: Got next segment. Inline: true Aug 7 17:59:56.825789608 (8h19m56.415134595s) [upload=8699]: Waiting for error group managing segments... Aug 7 17:59:56.825853870 (8h19m56.415198858s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:56.825858169 (8h19m56.415203146s) [upload=8699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:56.825860583 (8h19m56.41520557s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:57.105776123 (8h19m56.69512111s) [upload=8699]: Done uploading segments... Aug 7 17:59:57.105818965 (8h19m56.695163952s) [upload=8699]: Upload finished. err: <nil> Aug 7 18:01:03.618992056 (8h21m3.208337044s) [upload=8716]: Starting upload Aug 7 18:01:03.619055839 (8h21m3.208400826s) [upload=8716]: Done starting upload Aug 7 18:01:03.619137906 (8h21m3.208482893s) [upload=8716]: Uploading segments... Aug 7 18:01:03.619245863 (8h21m3.20859085s) [upload=8716]: Got next segment. Inline: true Aug 7 18:01:03.619250782 (8h21m3.208595769s) [upload=8716]: Waiting for error group managing segments... Aug 7 18:01:03.619294195 (8h21m3.208639182s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:03.619299145 (8h21m3.208644132s) [upload=8716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:01:03.619302010 (8h21m3.208646997s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:03.916050629 (8h21m3.505395616s) [upload=8716]: Done uploading segments... Aug 7 18:01:03.916069315 (8h21m3.505414302s) [upload=8716]: Upload finished. err: <nil> Aug 7 18:02:01.464319909 (8h22m1.053664906s) [upload=8733]: Starting upload Aug 7 18:02:01.464389803 (8h22m1.05373479s) [upload=8733]: Done starting upload Aug 7 18:02:01.464453204 (8h22m1.053798191s) [upload=8733]: Uploading segments... Aug 7 18:02:01.464502348 (8h22m1.053847335s) [upload=8733]: Got next segment. Inline: true Aug 7 18:02:01.464506175 (8h22m1.053851163s) [upload=8733]: Waiting for error group managing segments... Aug 7 18:02:01.464543306 (8h22m1.053888293s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:01.464547554 (8h22m1.053892541s) [upload=8733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:01.464550390 (8h22m1.053895378s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:01.743231809 (8h22m1.332576796s) [upload=8733]: Done uploading segments... Aug 7 18:02:01.743251787 (8h22m1.332596774s) [upload=8733]: Upload finished. err: <nil> Aug 7 18:02:42.581972343 (8h22m42.17131732s) [upload=8748]: Starting upload Aug 7 18:02:42.582022770 (8h22m42.171367747s) [upload=8748]: Done starting upload Aug 7 18:02:42.582068957 (8h22m42.171413945s) [upload=8748]: Uploading segments... Aug 7 18:02:42.582097883 (8h22m42.17144287s) [upload=8748]: Got next segment. Inline: true Aug 7 18:02:42.582102943 (8h22m42.17144793s) [upload=8748]: Waiting for error group managing segments... Aug 7 18:02:42.582136157 (8h22m42.171481134s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:42.582140294 (8h22m42.171485281s) [upload=8748]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:42.582142659 (8h22m42.171487646s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:42.866402070 (8h22m42.455747057s) [upload=8748]: Done uploading segments... Aug 7 18:02:42.866423541 (8h22m42.455768529s) [upload=8748]: Upload finished. err: <nil> Aug 7 18:02:58.324030621 (8h22m57.913375608s) [upload=8751]: Starting upload Aug 7 18:02:58.324074084 (8h22m57.913419071s) [upload=8751]: Done starting upload Aug 7 18:02:58.324126124 (8h22m57.913471101s) [upload=8751]: Uploading segments... Aug 7 18:02:58.324157975 (8h22m57.913502952s) [upload=8751]: Got next segment. Inline: true Aug 7 18:02:58.324164246 (8h22m57.913509233s) [upload=8751]: Waiting for error group managing segments... Aug 7 18:02:58.324196638 (8h22m57.913541625s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:58.324200235 (8h22m57.913545222s) [upload=8751]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Duration: 1.867540512s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............X............................XXX.......X..X...] duration: 5.686496038s FAIL (2025-08-07 18:06:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.686523091s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................................X..........X] duration: 1.633799897s FAIL (2025-08-07 18:06:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f931041cc3fd4f718673115b0ced3b75c502d0de3062da613e9263d5f1540ce9 != before d9d564b93c13f9f4e42038997ef8441abcb4d9316ab3b7b7504a89b217bb5e34 ======LOGS====== : pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:36.950552141 (8h19m36.539897128s) [upload=8693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:37.231511518 (8h19m36.820856505s) [upload=8693]: Done uploading segments... Aug 7 17:59:37.231531727 (8h19m36.820876714s) [upload=8693]: Upload finished. err: <nil> Aug 7 17:59:56.825653287 (8h19m56.414998274s) [upload=8699]: Starting upload Aug 7 17:59:56.825697972 (8h19m56.415042959s) [upload=8699]: Done starting upload Aug 7 17:59:56.825753909 (8h19m56.415098896s) [upload=8699]: Uploading segments... Aug 7 17:59:56.825786031 (8h19m56.415131018s) [upload=8699]: Got next segment. Inline: true Aug 7 17:59:56.825789608 (8h19m56.415134595s) [upload=8699]: Waiting for error group managing segments... Aug 7 17:59:56.825853870 (8h19m56.415198858s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:56.825858169 (8h19m56.415203146s) [upload=8699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:56.825860583 (8h19m56.41520557s) [upload=8699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:57.105776123 (8h19m56.69512111s) [upload=8699]: Done uploading segments... Aug 7 17:59:57.105818965 (8h19m56.695163952s) [upload=8699]: Upload finished. err: <nil> Aug 7 18:01:03.618992056 (8h21m3.208337044s) [upload=8716]: Starting upload Aug 7 18:01:03.619055839 (8h21m3.208400826s) [upload=8716]: Done starting upload Aug 7 18:01:03.619137906 (8h21m3.208482893s) [upload=8716]: Uploading segments... Aug 7 18:01:03.619245863 (8h21m3.20859085s) [upload=8716]: Got next segment. Inline: true Aug 7 18:01:03.619250782 (8h21m3.208595769s) [upload=8716]: Waiting for error group managing segments... Aug 7 18:01:03.619294195 (8h21m3.208639182s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:03.619299145 (8h21m3.208644132s) [upload=8716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:01:03.619302010 (8h21m3.208646997s) [upload=8716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:03.916050629 (8h21m3.505395616s) [upload=8716]: Done uploading segments... Aug 7 18:01:03.916069315 (8h21m3.505414302s) [upload=8716]: Upload finished. err: <nil> Aug 7 18:02:01.464319909 (8h22m1.053664906s) [upload=8733]: Starting upload Aug 7 18:02:01.464389803 (8h22m1.05373479s) [upload=8733]: Done starting upload Aug 7 18:02:01.464453204 (8h22m1.053798191s) [upload=8733]: Uploading segments... Aug 7 18:02:01.464502348 (8h22m1.053847335s) [upload=8733]: Got next segment. Inline: true Aug 7 18:02:01.464506175 (8h22m1.053851163s) [upload=8733]: Waiting for error group managing segments... Aug 7 18:02:01.464543306 (8h22m1.053888293s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:01.464547554 (8h22m1.053892541s) [upload=8733]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:01.464550390 (8h22m1.053895378s) [upload=8733]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:01.743231809 (8h22m1.332576796s) [upload=8733]: Done uploading segments... Aug 7 18:02:01.743251787 (8h22m1.332596774s) [upload=8733]: Upload finished. err: <nil> Aug 7 18:02:42.581972343 (8h22m42.17131732s) [upload=8748]: Starting upload Aug 7 18:02:42.582022770 (8h22m42.171367747s) [upload=8748]: Done starting upload Aug 7 18:02:42.582068957 (8h22m42.171413945s) [upload=8748]: Uploading segments... Aug 7 18:02:42.582097883 (8h22m42.17144287s) [upload=8748]: Got next segment. Inline: true Aug 7 18:02:42.582102943 (8h22m42.17144793s) [upload=8748]: Waiting for error group managing segments... Aug 7 18:02:42.582136157 (8h22m42.171481134s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:42.582140294 (8h22m42.171485281s) [upload=8748]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:42.582142659 (8h22m42.171487646s) [upload=8748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:42.866402070 (8h22m42.455747057s) [upload=8748]: Done uploading segments... Aug 7 18:02:42.866423541 (8h22m42.455768529s) [upload=8748]: Upload finished. err: <nil> Aug 7 18:02:58.324030621 (8h22m57.913375608s) [upload=8751]: Starting upload Aug 7 18:02:58.324074084 (8h22m57.913419071s) [upload=8751]: Done starting upload Aug 7 18:02:58.324126124 (8h22m57.913471101s) [upload=8751]: Uploading segments... Aug 7 18:02:58.324157975 (8h22m57.913502952s) [upload=8751]: Got next segment. Inline: true Aug 7 18:02:58.324164246 (8h22m57.913509233s) [upload=8751]: Waiting for error group managing segments... Aug 7 18:02:58.324196638 (8h22m57.913541625s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:02:58.324200235 (8h22m57.913545222s) [upload=8751]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:02:58.324202419 (8h22m57.913547406s) [upload=8751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:02:58.698491093 (8h22m58.28783608s) [upload=8751]: Done uploading segments... Aug 7 18:02:58.698512194 (8h22m58.287857181s) [upload=8751]: Upload finished. err: <nil> Aug 7 18:03:34.023713956 (8h23m33.613058943s) [upload=8762]: Starting upload Aug 7 18:03:34.023756718 (8h23m33.613101705s) [upload=8762]: Done starting upload Aug 7 18:03:34.023810802 (8h23m33.613155789s) [upload=8762]: Uploading segments... Aug 7 18:03:34.023849966 (8h23m33.613194953s) [upload=8762]: Got next segment. Inline: true Aug 7 18:03:34.023854666 (8h23m33.613199653s) [upload=8762]: Waiting for error group managing segments... Aug 7 18:03:34.023897628 (8h23m33.613242615s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:34.023901234 (8h23m33.613246211s) [upload=8762]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:34.023903569 (8h23m33.613248556s) [upload=8762]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:34.301041444 (8h23m33.890386431s) [upload=8762]: Done uploading segments... Aug 7 18:03:34.301059288 (8h23m33.890404275s) [upload=8762]: Upload finished. err: <nil> Aug 7 18:03:59.853483337 (8h23m59.442828324s) [upload=8770]: Starting upload Aug 7 18:03:59.853529155 (8h23m59.442874142s) [upload=8770]: Done starting upload Aug 7 18:03:59.853588008 (8h23m59.442932995s) [upload=8770]: Uploading segments... Aug 7 18:03:59.853685885 (8h23m59.443030902s) [upload=8770]: Got next segment. Inline: true Aug 7 18:03:59.853694351 (8h23m59.443039338s) [upload=8770]: Waiting for error group managing segments... Aug 7 18:03:59.853789563 (8h23m59.44313455s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:59.853811105 (8h23m59.443156092s) [upload=8770]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:03:59.853813669 (8h23m59.443158656s) [upload=8770]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:04:00.248702898 (8h23m59.838047905s) [upload=8770]: Done uploading segments... Aug 7 18:04:00.248729179 (8h23m59.838074166s) [upload=8770]: Upload finished. err: <nil> Aug 7 18:05:11.288723553 (8h25m10.87806854s) [upload=8792]: Starting upload Aug 7 18:05:11.288768570 (8h25m10.878113557s) [upload=8792]: Done starting upload Aug 7 18:05:11.288821371 (8h25m10.878166358s) [upload=8792]: Uploading segments... Aug 7 18:05:11.288853282 (8h25m10.878198269s) [upload=8792]: Got next segment. Inline: true Aug 7 18:05:11.288856989 (8h25m10.878201976s) [upload=8792]: Waiting for error group managing segments... Aug 7 18:05:11.288889871 (8h25m10.878234858s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:11.288894972 (8h25m10.878239969s) [upload=8792]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:11.288897246 (8h25m10.878242233s) [upload=8792]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:11.633494661 (8h25m11.222839648s) [upload=8792]: Done uploading segments... Aug 7 18:05:11.633511834 (8h25m11.222856821s) [upload=8792]: Upload finished. err: <nil> Aug 7 18:05:46.281330062 (8h25m45.870675059s) [upload=8802]: Starting upload Aug 7 18:05:46.281387371 (8h25m45.870732358s) [upload=8802]: Done starting upload Aug 7 18:05:46.281445092 (8h25m45.870790079s) [upload=8802]: Uploading segments... Aug 7 18:05:46.281475491 (8h25m45.870820478s) [upload=8802]: Got next segment. Inline: true Aug 7 18:05:46.281479218 (8h25m45.870824205s) [upload=8802]: Waiting for error group managing segments... Aug 7 18:05:46.281513543 (8h25m45.87085853s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:05:46.281517781 (8h25m45.870862768s) [upload=8802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:05:46.281520146 (8h25m45.870865133s) [upload=8802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:05:46.556872305 (8h25m46.146217292s) [upload=8802]: Done uploading segments... Aug 7 18:05:46.556891452 (8h25m46.146236439s) [upload=8802]: Upload finished. err: <nil> Aug 7 18:06:26.963610183 (8h26m26.55295517s) [upload=8814]: Starting upload Aug 7 18:06:26.963657103 (8h26m26.55300209s) [upload=8814]: Done starting upload Aug 7 18:06:26.963737527 (8h26m26.553082514s) [upload=8814]: Uploading segments... Aug 7 18:06:26.963867786 (8h26m26.553212783s) [upload=8814]: Got next segment. Inline: true Aug 7 18:06:26.963874910 (8h26m26.553219897s) [upload=8814]: Waiting for error group managing segments... Aug 7 18:06:26.963927700 (8h26m26.553272687s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:06:26.963934554 (8h26m26.553279541s) [upload=8814]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:06:26.963937970 (8h26m26.553282957s) [upload=8814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:06:27.258606837 (8h26m26.847951824s) [upload=8814]: Done uploading segments... Aug 7 18:06:27.258625452 (8h26m26.84797044s) [upload=8814]: Upload finished. err: <nil> Duration: 1.633823502s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............X............................XXX.......X..X..] duration: 5.984757212s FAIL (2025-08-07 18:05:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.98478264s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.845313283s FAIL (2025-08-07 18:05:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.845346767s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.9704359s FAIL (2025-08-07 18:05:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.970510983s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [................X............................XXX.......X..X.] duration: 5.782944955s FAIL (2025-08-07 18:04:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 5.782970363s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.997918752s FAIL (2025-08-07 18:04:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.997945163s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................X............................XXX.......X..X] duration: 869.894673ms FAIL (2025-08-07 18:03:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246329474 (8h23m5.835674461s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.246647382 (8h23m5.835992389s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247384402 (8h23m5.836729389s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247645822 (8h23m5.836990809s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247858339 (8h23m5.837203326s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.247887695 (8h23m5.837232682s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248302689 (8h23m5.837647676s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.248753101 (8h23m5.838098088s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249165820 (8h23m5.838510807s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.249591936 (8h23m5.838936923s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250046225 (8h23m5.839391212s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250465617 (8h23m5.839810604s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.250886683 (8h23m5.8402317s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.252128609 (8h23m5.841473596s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.253240015 (8h23m5.842585012s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:06.254658129 (8h23m5.844003156s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Uploading piece... Aug 7 18:03:07.364269399 (8h23m6.953614376s) [upload=8754]: Next returned nil segment Aug 7 18:03:07.364286913 (8h23m6.95363189s) [upload=8754]: Waiting for error group managing segments... Aug 7 18:03:07.673300135 (8h23m7.262645123s) [noise=true , address=64.49.13.140:4511 , piece_id=7HRZNC6JRVMWRLV65YAE3BQCDQBO75R2L5637AV5XWLY2BVMATKA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:07.692124972 (8h23m7.281469949s) [noise=true , address=64.49.13.26:4945 , piece_id=LICXRKOSSUHUSUIOPDJN7QZ5UXLZTFXA3F7DVQ7MK7ZLYL525DTQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184655888 (8h23m7.774000875s) [noise=true , address=104.36.82.55:4210 , piece_id=5BMNCME34WPD246GCZJQEVZXQED4C3KO52GCSVUXPLQQNK7AJYGA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.184675927 (8h23m7.774020914s) [noise=true , address=104.36.82.37:4672 , piece_id=BRRS5SZKB3F745QNDP67ESAEAFHWLBCKOPIH2Z3TUTQVJSFGUAAQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.210806672 (8h23m7.800151659s) [noise=true , address=104.36.82.45:4939 , piece_id=ZR2LKEL4W5UCX4XPAWSFGSGV73PEZSMUTNELPSKTYRSBNR5HKY5Q, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.241913799 (8h23m7.831258796s) [noise=true , address=104.36.82.8:4994 , piece_id=3HIFBLCQDKOUWWRV33T2UWCBHOYJPG4CVM4KEHLVWWSK4YDKYLAA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.254654997 (8h23m7.843999984s) [noise=true , address=104.36.82.5:4872 , piece_id=6SNZJCFCITZ4TOVALFDGXVKMIJRQ5F5OLQEBGH72PQH7S5DRPVNQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.277607612 (8h23m7.866952599s) [noise=true , address=104.36.82.56:4256 , piece_id=ZXESY6UGJTAAND35FXZR5LU4QNYQ6E6SQGSXTEUEPJVHQWGQMTGQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.289610297 (8h23m7.878955324s) [noise=true , address=104.36.82.42:4832 , piece_id=OBI3NLZ22AVSTC4UCHDRJNG57ZJB2JAOJOUR77EPZFKNMGVPD2OA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.311305837 (8h23m7.900650824s) [noise=true , address=104.36.82.35:4611 , piece_id=MQPXWU5W56AFEX7YB2FRMJQZW7XYGCJN5VVURGEKBE5PNJ7PKNFA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.336143870 (8h23m7.925488847s) [noise=true , address=104.36.82.59:4313 , piece_id=LKPVBLHJZHCLMO4MXJELKFJYJQOAARNK23QG7P7HY5PPEEPMPOOQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.341878110 (8h23m7.931223097s) [noise=true , address=104.36.82.46:4504 , piece_id=BNS7MP27RT3U7AVFQ7RS2VDSFW4XY6GEAIUBX3DGOU7CY5UN62BQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.351399628 (8h23m7.940744615s) [noise=true , address=104.36.82.16:4299 , piece_id=VWLFKNQD3IXL6N6RIDCPJRUTQUCBWTJ5TGQK6OPWQPUKOVTSGUZA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353409363 (8h23m7.94275434s) [noise=true , address=104.36.82.61:4382 , piece_id=DNC5RCHAFJC6HCSQEITZX2PZRXM63GSLVPLZPSDM5QTC3ABYDNVQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: <nil> Aug 7 18:03:08.353424572 (8h23m7.942769559s) [seg_pos={0 0}, upload=8754]: Segment reached optimal threshold of 14 pieces. Aug 7 18:03:08.353635446 (8h23m7.942980423s) [noise=true , address=104.36.82.60:4354 , piece_id=MKA5PMQM2GJKGS2BU223AAQCV2FTJ6FF57O5RI5AAVRSZMNLEXJA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353707804 (8h23m7.943052781s) [noise=true , address=45.91.116.8:4067 , piece_id=O6OOZG3SHQAGMG36XBF36P3GJHJ7LI5QYRD3RN4I3EEVC7QWV4JQ, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FDuuKhSWcQT3kRLyLd8g84Fu8VRZAwAKX3dLB4RjsXYmqdjTp): protocol: expected piece hash; context canceled; EOF Aug 7 18:03:08.353739585 (8h23m7.943084562s) [noise=true , address=104.36.82.23:4454 , piece_id=P5JYXDGZQWUEFJOMI2JUSLQP76LUBKOSTB5PJPFFWT3OWV5YBRBA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2): protocol: rpc: dial tcp 104.36.82.23:4454: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:03:08.353780403 (8h23m7.94312539s) [noise=true , address=104.36.82.58:4285 , piece_id=PNEQ5HY6XP6BOLXJVYJNSWRTL5D7P6QPBURS5ZVD2ALJFISFNRCA, seg_pos={0 0}, upload=8754]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp): protocol: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60120->104.36.82.58:4285: use of closed network connection Aug 7 18:03:08.353824899 (8h23m7.943169886s) [seg_pos={0 0}, upload=8754]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:03:08.353904971 (8h23m7.943249958s) [upload=8754]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:03:08.353908899 (8h23m7.943253876s) [upload=8754]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:03:08.643261075 (8h23m8.232606062s) [upload=8754]: Done uploading segments... Aug 7 18:03:08.643282024 (8h23m8.232627001s) [upload=8754]: Upload finished. err: <nil> Aug 7 18:03:50.182501705 (8h23m49.771846692s) [upload=8765]: Starting upload Aug 7 18:03:50.182547052 (8h23m49.771892039s) [upload=8765]: Done starting upload Aug 7 18:03:50.182714803 (8h23m49.77205979s) [upload=8765]: Uploading segments... Aug 7 18:03:50.182745772 (8h23m49.772090759s) [upload=8765]: Got next segment. Inline: false Aug 7 18:03:50.182770059 (8h23m49.772115046s) [upload=8765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:03:50.182774327 (8h23m49.772119314s) [upload=8765]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:03:50.409270941 (8h23m49.998615928s) [upload=8765]: Next segment err: metaclient: Too Many Requests Aug 7 18:03:50.409291371 (8h23m49.998636358s) [upload=8765]: Done uploading segments... Aug 7 18:03:50.409306419 (8h23m49.998651406s) [upload=8765]: Upload finished. err: metaclient: Too Many Requests Duration: 869.916995ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.003126689s FAIL (2025-08-07 18:03:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 2.00315339s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................X............................XXX.......X..] duration: 6.441568754s FAIL (2025-08-07 18:03:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== [upload=8720]: Done uploading segments... Aug 7 18:01:20.536011558 (8h21m20.125356546s) [upload=8720]: Upload finished. err: <nil> Aug 7 18:01:44.486402506 (8h21m44.075747503s) [upload=8728]: Starting upload Aug 7 18:01:44.486471799 (8h21m44.075816796s) [upload=8728]: Done starting upload Aug 7 18:01:44.486570116 (8h21m44.075915103s) [upload=8728]: Uploading segments... Aug 7 18:01:44.486752676 (8h21m44.076097673s) [upload=8728]: Got next segment. Inline: false Aug 7 18:01:44.486781401 (8h21m44.076126398s) [upload=8728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:44.486786781 (8h21m44.076131788s) [upload=8728]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:01:44.948685998 (8h21m44.538030975s) [seg_pos={0 0}, upload=8728]: Begin upload segment... Aug 7 18:01:44.948734019 (8h21m44.538078996s) [seg_pos={0 0}, upload=8728]: Done begin upload segment. Aug 7 18:01:44.948745903 (8h21m44.53809089s) [noise=true , address=104.36.82.61:4377 , piece_id=JO6VEUTIWKU54FCK27AJF6TBBOH4OWU32YURECYRRBVKXHBOB4IA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.948752745 (8h21m44.538097732s) [noise=true , address=104.36.82.43:4841 , piece_id=L6GSF5E3KVOJT3Y2CB5LEGCIKOAMIOPYJAXVBKBWWSIWPDE6QK6Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.948854711 (8h21m44.538199698s) [noise=true , address=64.49.13.155:4920 , piece_id=NRSMZXAETZD4OUTS4ZTLQPCSJIIQ2OK2HQPFG2RLUBJB6JBK67QA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.949111522 (8h21m44.538456499s) [noise=true , address=45.91.116.14:4192 , piece_id=KR7DPWHDRGJRUN5Y7RXRKW7FNW3TXCF5KW3DOWQ6JTU2ZRRQIK3Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.949616428 (8h21m44.538961415s) [noise=true , address=104.36.82.56:4255 , piece_id=JZWBENYZDESBRO4EZR3ZBDGL2BCZOHDP32QBVTQNLDPHZK7BWQWA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.950210825 (8h21m44.539555812s) [noise=true , address=104.36.82.5:4862 , piece_id=TQTH4TOXJKW7TDCT2SZ4M7NLEDY25Z7AON4GYXJDSYPP4DRQRZPA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.950300857 (8h21m44.539645844s) [noise=true , address=104.36.82.44:4892 , piece_id=GUHC4EQAHL7PH25LMKXY36QCAJEUZ7WU3MVNFTACU4DSUMKVDZFQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951055922 (8h21m44.540400909s) [noise=true , address=104.36.82.58:4290 , piece_id=HHRBA6B67YYZ2WYMLOXPZOXWE6C3AFBOTTMYOKE4Z6SQDPK4QDFQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951602699 (8h21m44.540947686s) [noise=true , address=104.36.82.46:4507 , piece_id=2I7GSX732NV67FVIFWRCBY2IUPEJW7626AD2TIFTH3WIM5VJN55Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951932610 (8h21m44.541277597s) [noise=true , address=104.36.82.60:4361 , piece_id=DNCCKV6NFDZMBLJGUF7QAOZGNSHJ2EBJFISPB5KBCJ3U27G4424A, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.952491328 (8h21m44.541836315s) [noise=true , address=104.36.82.62:4444 , piece_id=J2NZSZSFYX5ELLCG6ECBFE2VFQJTIOBZ6DJDCIY2V3I3ZK5DOVSQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.952819396 (8h21m44.542164373s) [noise=true , address=104.36.82.7:4924 , piece_id=3OWEJULXRCSBT2AODT2BTKTZJDU2BYIVWIJI32EWZYIO6D2JGWFA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953145360 (8h21m44.542490337s) [noise=true , address=45.91.116.5:4325 , piece_id=SXPHJTIQZTNQZYT5M6TK6JKJY2DJ2DLVP46UEQLFYF6KYZRVTAWA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953574390 (8h21m44.542919377s) [noise=true , address=104.36.82.47:4521 , piece_id=VMWVPITIBDQX25N4XUKHRA3GADSWSWQGVB2DKHZCEX3M6VHDHZUQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953912838 (8h21m44.543257826s) [noise=true , address=64.49.13.26:4948 , piece_id=HANRWGVLIPHOSX3JGHMKLEIP7D446L34S6RZJCETVUCWDII22VSA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.954757866 (8h21m44.544102853s) [noise=true , address=79.127.205.233:4647 , piece_id=G26VBRMOT4GWIRGIJLB3SYMW77PEKFZL5Z3ZFULRBZPKGE3ZD22A, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.955027871 (8h21m44.544372858s) [noise=true , address=104.36.82.2:4753 , piece_id=Y4GQNY4OAZ5BB7CKDJQI6QG2Z33B3GSLPADXZHGSFUXM5CB7TOUA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.955298349 (8h21m44.544643336s) [noise=true , address=104.36.82.23:4464 , piece_id=UPQK73PHM3OFOLYHYU5X55P4K5HKDU5ZZ3YXGA4KLA22GLENXBLA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:46.056497965 (8h21m45.645842952s) [upload=8728]: Next returned nil segment Aug 7 18:01:46.056513445 (8h21m45.645858422s) [upload=8728]: Waiting for error group managing segments... Aug 7 18:01:46.337246008 (8h21m45.926590985s) [noise=true , address=79.127.205.233:4647 , piece_id=G26VBRMOT4GWIRGIJLB3SYMW77PEKFZL5Z3ZFULRBZPKGE3ZD22A, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.454140462 (8h21m46.043485439s) [noise=true , address=64.49.13.155:4920 , piece_id=NRSMZXAETZD4OUTS4ZTLQPCSJIIQ2OK2HQPFG2RLUBJB6JBK67QA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.487355753 (8h21m46.07670074s) [noise=true , address=64.49.13.26:4948 , piece_id=HANRWGVLIPHOSX3JGHMKLEIP7D446L34S6RZJCETVUCWDII22VSA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.841410603 (8h21m46.43075559s) [noise=true , address=45.91.116.5:4325 , piece_id=SXPHJTIQZTNQZYT5M6TK6JKJY2DJ2DLVP46UEQLFYF6KYZRVTAWA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.859997515 (8h21m46.449342492s) [noise=true , address=104.36.82.60:4361 , piece_id=DNCCKV6NFDZMBLJGUF7QAOZGNSHJ2EBJFISPB5KBCJ3U27G4424A, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.917035595 (8h21m46.506380572s) [noise=true , address=104.36.82.47:4521 , piece_id=VMWVPITIBDQX25N4XUKHRA3GADSWSWQGVB2DKHZCEX3M6VHDHZUQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.930116233 (8h21m46.51946122s) [noise=true , address=104.36.82.44:4892 , piece_id=GUHC4EQAHL7PH25LMKXY36QCAJEUZ7WU3MVNFTACU4DSUMKVDZFQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.943325155 (8h21m46.532670132s) [noise=true , address=104.36.82.58:4290 , piece_id=HHRBA6B67YYZ2WYMLOXPZOXWE6C3AFBOTTMYOKE4Z6SQDPK4QDFQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.951644862 (8h21m46.540989839s) [noise=true , address=104.36.82.23:4464 , piece_id=UPQK73PHM3OFOLYHYU5X55P4K5HKDU5ZZ3YXGA4KLA22GLENXBLA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.956182053 (8h21m46.54552704s) [noise=true , address=104.36.82.7:4924 , piece_id=3OWEJULXRCSBT2AODT2BTKTZJDU2BYIVWIJI32EWZYIO6D2JGWFA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.958069505 (8h21m46.547414482s) [noise=true , address=104.36.82.46:4507 , piece_id=2I7GSX732NV67FVIFWRCBY2IUPEJW7626AD2TIFTH3WIM5VJN55Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.959327923 (8h21m46.54867291s) [noise=true , address=45.91.116.14:4192 , piece_id=KR7DPWHDRGJRUN5Y7RXRKW7FNW3TXCF5KW3DOWQ6JTU2ZRRQIK3Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.976827544 (8h21m46.566172522s) [noise=true , address=104.36.82.56:4255 , piece_id=JZWBENYZDESBRO4EZR3ZBDGL2BCZOHDP32QBVTQNLDPHZK7BWQWA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.980770710 (8h21m46.570115697s) [noise=true , address=104.36.82.62:4444 , piece_id=J2NZSZSFYX5ELLCG6ECBFE2VFQJTIOBZ6DJDCIY2V3I3ZK5DOVSQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.980789786 (8h21m46.570134773s) [seg_pos={0 0}, upload=8728]: Segment reached optimal threshold of 14 pieces. Aug 7 18:01:46.980895608 (8h21m46.570240595s) [noise=true , address=104.36.82.2:4753 , piece_id=Y4GQNY4OAZ5BB7CKDJQI6QG2Z33B3GSLPADXZHGSFUXM5CB7TOUA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.980948990 (8h21m46.570293967s) [noise=true , address=104.36.82.43:4841 , piece_id=L6GSF5E3KVOJT3Y2CB5LEGCIKOAMIOPYJAXVBKBWWSIWPDE6QK6Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981039143 (8h21m46.57038412s) [noise=true , address=104.36.82.61:4377 , piece_id=JO6VEUTIWKU54FCK27AJF6TBBOH4OWU32YURECYRRBVKXHBOB4IA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126HKpC7sciWFTxNaH5Hjs6BBuh8NRgu8fA4cu4qcK29zfJWg7R): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981052468 (8h21m46.570397455s) [noise=true , address=104.36.82.5:4862 , piece_id=TQTH4TOXJKW7TDCT2SZ4M7NLEDY25Z7AON4GYXJDSYPP4DRQRZPA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981089649 (8h21m46.570434626s) [seg_pos={0 0}, upload=8728]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:01:46.981181115 (8h21m46.570526102s) [upload=8728]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:01:46.981185663 (8h21m46.57053065s) [upload=8728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:47.287909635 (8h21m46.877254622s) [upload=8728]: Done uploading segments... Aug 7 18:01:47.287927600 (8h21m46.877272587s) [upload=8728]: Upload finished. err: <nil> Duration: 6.44159804s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.99620295s FAIL (2025-08-07 18:02:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.99622961s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.87089554s FAIL (2025-08-07 18:02:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.870922082s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................X............................XXX.......X.] duration: 5.95911162s FAIL (2025-08-07 18:02:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== [upload=8720]: Done uploading segments... Aug 7 18:01:20.536011558 (8h21m20.125356546s) [upload=8720]: Upload finished. err: <nil> Aug 7 18:01:44.486402506 (8h21m44.075747503s) [upload=8728]: Starting upload Aug 7 18:01:44.486471799 (8h21m44.075816796s) [upload=8728]: Done starting upload Aug 7 18:01:44.486570116 (8h21m44.075915103s) [upload=8728]: Uploading segments... Aug 7 18:01:44.486752676 (8h21m44.076097673s) [upload=8728]: Got next segment. Inline: false Aug 7 18:01:44.486781401 (8h21m44.076126398s) [upload=8728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:44.486786781 (8h21m44.076131788s) [upload=8728]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:01:44.948685998 (8h21m44.538030975s) [seg_pos={0 0}, upload=8728]: Begin upload segment... Aug 7 18:01:44.948734019 (8h21m44.538078996s) [seg_pos={0 0}, upload=8728]: Done begin upload segment. Aug 7 18:01:44.948745903 (8h21m44.53809089s) [noise=true , address=104.36.82.61:4377 , piece_id=JO6VEUTIWKU54FCK27AJF6TBBOH4OWU32YURECYRRBVKXHBOB4IA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.948752745 (8h21m44.538097732s) [noise=true , address=104.36.82.43:4841 , piece_id=L6GSF5E3KVOJT3Y2CB5LEGCIKOAMIOPYJAXVBKBWWSIWPDE6QK6Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.948854711 (8h21m44.538199698s) [noise=true , address=64.49.13.155:4920 , piece_id=NRSMZXAETZD4OUTS4ZTLQPCSJIIQ2OK2HQPFG2RLUBJB6JBK67QA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.949111522 (8h21m44.538456499s) [noise=true , address=45.91.116.14:4192 , piece_id=KR7DPWHDRGJRUN5Y7RXRKW7FNW3TXCF5KW3DOWQ6JTU2ZRRQIK3Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.949616428 (8h21m44.538961415s) [noise=true , address=104.36.82.56:4255 , piece_id=JZWBENYZDESBRO4EZR3ZBDGL2BCZOHDP32QBVTQNLDPHZK7BWQWA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.950210825 (8h21m44.539555812s) [noise=true , address=104.36.82.5:4862 , piece_id=TQTH4TOXJKW7TDCT2SZ4M7NLEDY25Z7AON4GYXJDSYPP4DRQRZPA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.950300857 (8h21m44.539645844s) [noise=true , address=104.36.82.44:4892 , piece_id=GUHC4EQAHL7PH25LMKXY36QCAJEUZ7WU3MVNFTACU4DSUMKVDZFQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951055922 (8h21m44.540400909s) [noise=true , address=104.36.82.58:4290 , piece_id=HHRBA6B67YYZ2WYMLOXPZOXWE6C3AFBOTTMYOKE4Z6SQDPK4QDFQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951602699 (8h21m44.540947686s) [noise=true , address=104.36.82.46:4507 , piece_id=2I7GSX732NV67FVIFWRCBY2IUPEJW7626AD2TIFTH3WIM5VJN55Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951932610 (8h21m44.541277597s) [noise=true , address=104.36.82.60:4361 , piece_id=DNCCKV6NFDZMBLJGUF7QAOZGNSHJ2EBJFISPB5KBCJ3U27G4424A, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.952491328 (8h21m44.541836315s) [noise=true , address=104.36.82.62:4444 , piece_id=J2NZSZSFYX5ELLCG6ECBFE2VFQJTIOBZ6DJDCIY2V3I3ZK5DOVSQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.952819396 (8h21m44.542164373s) [noise=true , address=104.36.82.7:4924 , piece_id=3OWEJULXRCSBT2AODT2BTKTZJDU2BYIVWIJI32EWZYIO6D2JGWFA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953145360 (8h21m44.542490337s) [noise=true , address=45.91.116.5:4325 , piece_id=SXPHJTIQZTNQZYT5M6TK6JKJY2DJ2DLVP46UEQLFYF6KYZRVTAWA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953574390 (8h21m44.542919377s) [noise=true , address=104.36.82.47:4521 , piece_id=VMWVPITIBDQX25N4XUKHRA3GADSWSWQGVB2DKHZCEX3M6VHDHZUQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953912838 (8h21m44.543257826s) [noise=true , address=64.49.13.26:4948 , piece_id=HANRWGVLIPHOSX3JGHMKLEIP7D446L34S6RZJCETVUCWDII22VSA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.954757866 (8h21m44.544102853s) [noise=true , address=79.127.205.233:4647 , piece_id=G26VBRMOT4GWIRGIJLB3SYMW77PEKFZL5Z3ZFULRBZPKGE3ZD22A, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.955027871 (8h21m44.544372858s) [noise=true , address=104.36.82.2:4753 , piece_id=Y4GQNY4OAZ5BB7CKDJQI6QG2Z33B3GSLPADXZHGSFUXM5CB7TOUA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.955298349 (8h21m44.544643336s) [noise=true , address=104.36.82.23:4464 , piece_id=UPQK73PHM3OFOLYHYU5X55P4K5HKDU5ZZ3YXGA4KLA22GLENXBLA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:46.056497965 (8h21m45.645842952s) [upload=8728]: Next returned nil segment Aug 7 18:01:46.056513445 (8h21m45.645858422s) [upload=8728]: Waiting for error group managing segments... Aug 7 18:01:46.337246008 (8h21m45.926590985s) [noise=true , address=79.127.205.233:4647 , piece_id=G26VBRMOT4GWIRGIJLB3SYMW77PEKFZL5Z3ZFULRBZPKGE3ZD22A, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.454140462 (8h21m46.043485439s) [noise=true , address=64.49.13.155:4920 , piece_id=NRSMZXAETZD4OUTS4ZTLQPCSJIIQ2OK2HQPFG2RLUBJB6JBK67QA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.487355753 (8h21m46.07670074s) [noise=true , address=64.49.13.26:4948 , piece_id=HANRWGVLIPHOSX3JGHMKLEIP7D446L34S6RZJCETVUCWDII22VSA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.841410603 (8h21m46.43075559s) [noise=true , address=45.91.116.5:4325 , piece_id=SXPHJTIQZTNQZYT5M6TK6JKJY2DJ2DLVP46UEQLFYF6KYZRVTAWA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.859997515 (8h21m46.449342492s) [noise=true , address=104.36.82.60:4361 , piece_id=DNCCKV6NFDZMBLJGUF7QAOZGNSHJ2EBJFISPB5KBCJ3U27G4424A, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.917035595 (8h21m46.506380572s) [noise=true , address=104.36.82.47:4521 , piece_id=VMWVPITIBDQX25N4XUKHRA3GADSWSWQGVB2DKHZCEX3M6VHDHZUQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.930116233 (8h21m46.51946122s) [noise=true , address=104.36.82.44:4892 , piece_id=GUHC4EQAHL7PH25LMKXY36QCAJEUZ7WU3MVNFTACU4DSUMKVDZFQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.943325155 (8h21m46.532670132s) [noise=true , address=104.36.82.58:4290 , piece_id=HHRBA6B67YYZ2WYMLOXPZOXWE6C3AFBOTTMYOKE4Z6SQDPK4QDFQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.951644862 (8h21m46.540989839s) [noise=true , address=104.36.82.23:4464 , piece_id=UPQK73PHM3OFOLYHYU5X55P4K5HKDU5ZZ3YXGA4KLA22GLENXBLA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.956182053 (8h21m46.54552704s) [noise=true , address=104.36.82.7:4924 , piece_id=3OWEJULXRCSBT2AODT2BTKTZJDU2BYIVWIJI32EWZYIO6D2JGWFA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.958069505 (8h21m46.547414482s) [noise=true , address=104.36.82.46:4507 , piece_id=2I7GSX732NV67FVIFWRCBY2IUPEJW7626AD2TIFTH3WIM5VJN55Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.959327923 (8h21m46.54867291s) [noise=true , address=45.91.116.14:4192 , piece_id=KR7DPWHDRGJRUN5Y7RXRKW7FNW3TXCF5KW3DOWQ6JTU2ZRRQIK3Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.976827544 (8h21m46.566172522s) [noise=true , address=104.36.82.56:4255 , piece_id=JZWBENYZDESBRO4EZR3ZBDGL2BCZOHDP32QBVTQNLDPHZK7BWQWA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.980770710 (8h21m46.570115697s) [noise=true , address=104.36.82.62:4444 , piece_id=J2NZSZSFYX5ELLCG6ECBFE2VFQJTIOBZ6DJDCIY2V3I3ZK5DOVSQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.980789786 (8h21m46.570134773s) [seg_pos={0 0}, upload=8728]: Segment reached optimal threshold of 14 pieces. Aug 7 18:01:46.980895608 (8h21m46.570240595s) [noise=true , address=104.36.82.2:4753 , piece_id=Y4GQNY4OAZ5BB7CKDJQI6QG2Z33B3GSLPADXZHGSFUXM5CB7TOUA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.980948990 (8h21m46.570293967s) [noise=true , address=104.36.82.43:4841 , piece_id=L6GSF5E3KVOJT3Y2CB5LEGCIKOAMIOPYJAXVBKBWWSIWPDE6QK6Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981039143 (8h21m46.57038412s) [noise=true , address=104.36.82.61:4377 , piece_id=JO6VEUTIWKU54FCK27AJF6TBBOH4OWU32YURECYRRBVKXHBOB4IA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126HKpC7sciWFTxNaH5Hjs6BBuh8NRgu8fA4cu4qcK29zfJWg7R): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981052468 (8h21m46.570397455s) [noise=true , address=104.36.82.5:4862 , piece_id=TQTH4TOXJKW7TDCT2SZ4M7NLEDY25Z7AON4GYXJDSYPP4DRQRZPA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981089649 (8h21m46.570434626s) [seg_pos={0 0}, upload=8728]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:01:46.981181115 (8h21m46.570526102s) [upload=8728]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:01:46.981185663 (8h21m46.57053065s) [upload=8728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:47.287909635 (8h21m46.877254622s) [upload=8728]: Done uploading segments... Aug 7 18:01:47.287927600 (8h21m46.877272587s) [upload=8728]: Upload finished. err: <nil> Duration: 5.959142379s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.919912254s FAIL (2025-08-07 18:02:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.919933754s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................X............................XXX.......X] duration: 3.894452845s FAIL (2025-08-07 18:01:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== [upload=8720]: Done uploading segments... Aug 7 18:01:20.536011558 (8h21m20.125356546s) [upload=8720]: Upload finished. err: <nil> Aug 7 18:01:44.486402506 (8h21m44.075747503s) [upload=8728]: Starting upload Aug 7 18:01:44.486471799 (8h21m44.075816796s) [upload=8728]: Done starting upload Aug 7 18:01:44.486570116 (8h21m44.075915103s) [upload=8728]: Uploading segments... Aug 7 18:01:44.486752676 (8h21m44.076097673s) [upload=8728]: Got next segment. Inline: false Aug 7 18:01:44.486781401 (8h21m44.076126398s) [upload=8728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:01:44.486786781 (8h21m44.076131788s) [upload=8728]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:01:44.948685998 (8h21m44.538030975s) [seg_pos={0 0}, upload=8728]: Begin upload segment... Aug 7 18:01:44.948734019 (8h21m44.538078996s) [seg_pos={0 0}, upload=8728]: Done begin upload segment. Aug 7 18:01:44.948745903 (8h21m44.53809089s) [noise=true , address=104.36.82.61:4377 , piece_id=JO6VEUTIWKU54FCK27AJF6TBBOH4OWU32YURECYRRBVKXHBOB4IA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.948752745 (8h21m44.538097732s) [noise=true , address=104.36.82.43:4841 , piece_id=L6GSF5E3KVOJT3Y2CB5LEGCIKOAMIOPYJAXVBKBWWSIWPDE6QK6Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.948854711 (8h21m44.538199698s) [noise=true , address=64.49.13.155:4920 , piece_id=NRSMZXAETZD4OUTS4ZTLQPCSJIIQ2OK2HQPFG2RLUBJB6JBK67QA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.949111522 (8h21m44.538456499s) [noise=true , address=45.91.116.14:4192 , piece_id=KR7DPWHDRGJRUN5Y7RXRKW7FNW3TXCF5KW3DOWQ6JTU2ZRRQIK3Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.949616428 (8h21m44.538961415s) [noise=true , address=104.36.82.56:4255 , piece_id=JZWBENYZDESBRO4EZR3ZBDGL2BCZOHDP32QBVTQNLDPHZK7BWQWA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.950210825 (8h21m44.539555812s) [noise=true , address=104.36.82.5:4862 , piece_id=TQTH4TOXJKW7TDCT2SZ4M7NLEDY25Z7AON4GYXJDSYPP4DRQRZPA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.950300857 (8h21m44.539645844s) [noise=true , address=104.36.82.44:4892 , piece_id=GUHC4EQAHL7PH25LMKXY36QCAJEUZ7WU3MVNFTACU4DSUMKVDZFQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951055922 (8h21m44.540400909s) [noise=true , address=104.36.82.58:4290 , piece_id=HHRBA6B67YYZ2WYMLOXPZOXWE6C3AFBOTTMYOKE4Z6SQDPK4QDFQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951602699 (8h21m44.540947686s) [noise=true , address=104.36.82.46:4507 , piece_id=2I7GSX732NV67FVIFWRCBY2IUPEJW7626AD2TIFTH3WIM5VJN55Q, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.951932610 (8h21m44.541277597s) [noise=true , address=104.36.82.60:4361 , piece_id=DNCCKV6NFDZMBLJGUF7QAOZGNSHJ2EBJFISPB5KBCJ3U27G4424A, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.952491328 (8h21m44.541836315s) [noise=true , address=104.36.82.62:4444 , piece_id=J2NZSZSFYX5ELLCG6ECBFE2VFQJTIOBZ6DJDCIY2V3I3ZK5DOVSQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.952819396 (8h21m44.542164373s) [noise=true , address=104.36.82.7:4924 , piece_id=3OWEJULXRCSBT2AODT2BTKTZJDU2BYIVWIJI32EWZYIO6D2JGWFA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953145360 (8h21m44.542490337s) [noise=true , address=45.91.116.5:4325 , piece_id=SXPHJTIQZTNQZYT5M6TK6JKJY2DJ2DLVP46UEQLFYF6KYZRVTAWA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953574390 (8h21m44.542919377s) [noise=true , address=104.36.82.47:4521 , piece_id=VMWVPITIBDQX25N4XUKHRA3GADSWSWQGVB2DKHZCEX3M6VHDHZUQ, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.953912838 (8h21m44.543257826s) [noise=true , address=64.49.13.26:4948 , piece_id=HANRWGVLIPHOSX3JGHMKLEIP7D446L34S6RZJCETVUCWDII22VSA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.954757866 (8h21m44.544102853s) [noise=true , address=79.127.205.233:4647 , piece_id=G26VBRMOT4GWIRGIJLB3SYMW77PEKFZL5Z3ZFULRBZPKGE3ZD22A, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.955027871 (8h21m44.544372858s) [noise=true , address=104.36.82.2:4753 , piece_id=Y4GQNY4OAZ5BB7CKDJQI6QG2Z33B3GSLPADXZHGSFUXM5CB7TOUA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:44.955298349 (8h21m44.544643336s) [noise=true , address=104.36.82.23:4464 , piece_id=UPQK73PHM3OFOLYHYU5X55P4K5HKDU5ZZ3YXGA4KLA22GLENXBLA, seg_pos={0 0}, upload=8728]: Uploading piece... Aug 7 18:01:46.056497965 (8h21m45.645842952s) [upload=8728]: Next returned nil segment Aug 7 18:01:46.056513445 (8h21m45.645858422s) [upload=8728]: Waiting for error group managing segments... Aug 7 18:01:46.337246008 (8h21m45.926590985s) [noise=true , address=79.127.205.233:4647 , piece_id=G26VBRMOT4GWIRGIJLB3SYMW77PEKFZL5Z3ZFULRBZPKGE3ZD22A, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.454140462 (8h21m46.043485439s) [noise=true , address=64.49.13.155:4920 , piece_id=NRSMZXAETZD4OUTS4ZTLQPCSJIIQ2OK2HQPFG2RLUBJB6JBK67QA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.487355753 (8h21m46.07670074s) [noise=true , address=64.49.13.26:4948 , piece_id=HANRWGVLIPHOSX3JGHMKLEIP7D446L34S6RZJCETVUCWDII22VSA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.841410603 (8h21m46.43075559s) [noise=true , address=45.91.116.5:4325 , piece_id=SXPHJTIQZTNQZYT5M6TK6JKJY2DJ2DLVP46UEQLFYF6KYZRVTAWA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.859997515 (8h21m46.449342492s) [noise=true , address=104.36.82.60:4361 , piece_id=DNCCKV6NFDZMBLJGUF7QAOZGNSHJ2EBJFISPB5KBCJ3U27G4424A, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.917035595 (8h21m46.506380572s) [noise=true , address=104.36.82.47:4521 , piece_id=VMWVPITIBDQX25N4XUKHRA3GADSWSWQGVB2DKHZCEX3M6VHDHZUQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.930116233 (8h21m46.51946122s) [noise=true , address=104.36.82.44:4892 , piece_id=GUHC4EQAHL7PH25LMKXY36QCAJEUZ7WU3MVNFTACU4DSUMKVDZFQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.943325155 (8h21m46.532670132s) [noise=true , address=104.36.82.58:4290 , piece_id=HHRBA6B67YYZ2WYMLOXPZOXWE6C3AFBOTTMYOKE4Z6SQDPK4QDFQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.951644862 (8h21m46.540989839s) [noise=true , address=104.36.82.23:4464 , piece_id=UPQK73PHM3OFOLYHYU5X55P4K5HKDU5ZZ3YXGA4KLA22GLENXBLA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.956182053 (8h21m46.54552704s) [noise=true , address=104.36.82.7:4924 , piece_id=3OWEJULXRCSBT2AODT2BTKTZJDU2BYIVWIJI32EWZYIO6D2JGWFA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.958069505 (8h21m46.547414482s) [noise=true , address=104.36.82.46:4507 , piece_id=2I7GSX732NV67FVIFWRCBY2IUPEJW7626AD2TIFTH3WIM5VJN55Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.959327923 (8h21m46.54867291s) [noise=true , address=45.91.116.14:4192 , piece_id=KR7DPWHDRGJRUN5Y7RXRKW7FNW3TXCF5KW3DOWQ6JTU2ZRRQIK3Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.976827544 (8h21m46.566172522s) [noise=true , address=104.36.82.56:4255 , piece_id=JZWBENYZDESBRO4EZR3ZBDGL2BCZOHDP32QBVTQNLDPHZK7BWQWA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.980770710 (8h21m46.570115697s) [noise=true , address=104.36.82.62:4444 , piece_id=J2NZSZSFYX5ELLCG6ECBFE2VFQJTIOBZ6DJDCIY2V3I3ZK5DOVSQ, seg_pos={0 0}, upload=8728]: Done uploading piece. err: <nil> Aug 7 18:01:46.980789786 (8h21m46.570134773s) [seg_pos={0 0}, upload=8728]: Segment reached optimal threshold of 14 pieces. Aug 7 18:01:46.980895608 (8h21m46.570240595s) [noise=true , address=104.36.82.2:4753 , piece_id=Y4GQNY4OAZ5BB7CKDJQI6QG2Z33B3GSLPADXZHGSFUXM5CB7TOUA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.980948990 (8h21m46.570293967s) [noise=true , address=104.36.82.43:4841 , piece_id=L6GSF5E3KVOJT3Y2CB5LEGCIKOAMIOPYJAXVBKBWWSIWPDE6QK6Q, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FKz4ioR4bYX3jEHXQAUGeJwhCxzVtXfA3kddFrb84YNpJeRQD): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981039143 (8h21m46.57038412s) [noise=true , address=104.36.82.61:4377 , piece_id=JO6VEUTIWKU54FCK27AJF6TBBOH4OWU32YURECYRRBVKXHBOB4IA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126HKpC7sciWFTxNaH5Hjs6BBuh8NRgu8fA4cu4qcK29zfJWg7R): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981052468 (8h21m46.570397455s) [noise=true , address=104.36.82.5:4862 , piece_id=TQTH4TOXJKW7TDCT2SZ4M7NLEDY25Z7AON4GYXJDSYPP4DRQRZPA, seg_pos={0 0}, upload=8728]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue): protocol: expected piece hash; context canceled; EOF Aug 7 18:01:46.981089649 (8h21m46.570434626s) [seg_pos={0 0}, upload=8728]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:01:46.981181115 (8h21m46.570526102s) [upload=8728]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:01:46.981185663 (8h21m46.57053065s) [upload=8728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:01:47.287909635 (8h21m46.877254622s) [upload=8728]: Done uploading segments... Aug 7 18:01:47.287927600 (8h21m46.877272587s) [upload=8728]: Upload finished. err: <nil> Duration: 3.894477573s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X............................XXX.......] duration: 5.982109754s FAIL (2025-08-07 18:01:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== load=8629]: Done uploading segments... Aug 7 17:56:00.782013583 (8h16m0.37135857s) [upload=8629]: Upload finished. err: <nil> Aug 7 17:56:38.723311581 (8h16m38.312656568s) [upload=8641]: Starting upload Aug 7 17:56:38.723381295 (8h16m38.312726282s) [upload=8641]: Done starting upload Aug 7 17:56:38.723510432 (8h16m38.312855419s) [upload=8641]: Uploading segments... Aug 7 17:56:38.723689655 (8h16m38.313034642s) [upload=8641]: Got next segment. Inline: false Aug 7 17:56:38.723742636 (8h16m38.313087624s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:38.723747806 (8h16m38.313092813s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:56:39.185110639 (8h16m38.774455626s) [seg_pos={0 0}, upload=8641]: Begin upload segment... Aug 7 17:56:39.185155283 (8h16m38.77450027s) [seg_pos={0 0}, upload=8641]: Done begin upload segment. Aug 7 17:56:39.185179680 (8h16m38.774524657s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185283429 (8h16m38.774628416s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185591228 (8h16m38.774936215s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185791812 (8h16m38.775136809s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185997285 (8h16m38.775342272s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186141400 (8h16m38.775486387s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186247333 (8h16m38.77559232s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186429762 (8h16m38.775774739s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186588346 (8h16m38.775933333s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186724576 (8h16m38.776069563s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186842933 (8h16m38.77618791s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187031925 (8h16m38.776376912s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187135412 (8h16m38.776480399s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187272524 (8h16m38.776617511s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187389569 (8h16m38.776734546s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187506292 (8h16m38.776851279s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187618787 (8h16m38.776963774s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187887912 (8h16m38.777232889s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:40.297071894 (8h16m39.886416881s) [upload=8641]: Next returned nil segment Aug 7 17:56:40.297094217 (8h16m39.886439204s) [upload=8641]: Waiting for error group managing segments... Aug 7 17:56:40.607636446 (8h16m40.196981433s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.119004102 (8h16m40.708349089s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.138326000 (8h16m40.727670977s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.143987792 (8h16m40.733332779s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.145683708 (8h16m40.735028695s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.179729887 (8h16m40.769074874s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.184258271 (8h16m40.773603258s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189182012 (8h16m40.778526989s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189550136 (8h16m40.778895123s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.190054842 (8h16m40.779399829s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.197377522 (8h16m40.786722509s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.203901384 (8h16m40.793246411s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.212457535 (8h16m40.801802522s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218587324 (8h16m40.807932311s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218619645 (8h16m40.807964662s) [seg_pos={0 0}, upload=8641]: Segment reached optimal threshold of 14 pieces. Aug 7 17:56:41.218795952 (8h16m40.808140939s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1251oUFjESKjnTubb2wF8K9nsa8fztBm4nup3GvAymHLExfH35v): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218830829 (8h16m40.808175806s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ec4Xvp3nGRAm9BivLLTp6AVkMoz1i2AgXQdfQaAmNPLHWD4Yv): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218923186 (8h16m40.808268233s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gJKrCBLJWN1SBKibfeqBtcqqiv4veKTGmtnycjZyb3VN9kwoT): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218971639 (8h16m40.808316626s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b1UCF4dQobQEVqe9XB5tKxDx9L2P7kkQW179fxc1ir9ZtQVsM): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218999302 (8h16m40.808344279s) [seg_pos={0 0}, upload=8641]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:56:41.219090676 (8h16m40.808435663s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:56:41.219094864 (8h16m40.808439851s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:41.533326972 (8h16m41.122671949s) [upload=8641]: Done uploading segments... Aug 7 17:56:41.533346088 (8h16m41.122691065s) [upload=8641]: Upload finished. err: <nil> Duration: 5.982136016s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.003046191s FAIL (2025-08-07 18:01:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 2.003088152s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X............................XXX......] duration: 6.313558532s FAIL (2025-08-07 18:00:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== load=8629]: Done uploading segments... Aug 7 17:56:00.782013583 (8h16m0.37135857s) [upload=8629]: Upload finished. err: <nil> Aug 7 17:56:38.723311581 (8h16m38.312656568s) [upload=8641]: Starting upload Aug 7 17:56:38.723381295 (8h16m38.312726282s) [upload=8641]: Done starting upload Aug 7 17:56:38.723510432 (8h16m38.312855419s) [upload=8641]: Uploading segments... Aug 7 17:56:38.723689655 (8h16m38.313034642s) [upload=8641]: Got next segment. Inline: false Aug 7 17:56:38.723742636 (8h16m38.313087624s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:38.723747806 (8h16m38.313092813s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:56:39.185110639 (8h16m38.774455626s) [seg_pos={0 0}, upload=8641]: Begin upload segment... Aug 7 17:56:39.185155283 (8h16m38.77450027s) [seg_pos={0 0}, upload=8641]: Done begin upload segment. Aug 7 17:56:39.185179680 (8h16m38.774524657s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185283429 (8h16m38.774628416s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185591228 (8h16m38.774936215s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185791812 (8h16m38.775136809s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185997285 (8h16m38.775342272s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186141400 (8h16m38.775486387s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186247333 (8h16m38.77559232s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186429762 (8h16m38.775774739s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186588346 (8h16m38.775933333s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186724576 (8h16m38.776069563s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186842933 (8h16m38.77618791s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187031925 (8h16m38.776376912s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187135412 (8h16m38.776480399s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187272524 (8h16m38.776617511s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187389569 (8h16m38.776734546s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187506292 (8h16m38.776851279s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187618787 (8h16m38.776963774s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187887912 (8h16m38.777232889s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:40.297071894 (8h16m39.886416881s) [upload=8641]: Next returned nil segment Aug 7 17:56:40.297094217 (8h16m39.886439204s) [upload=8641]: Waiting for error group managing segments... Aug 7 17:56:40.607636446 (8h16m40.196981433s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.119004102 (8h16m40.708349089s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.138326000 (8h16m40.727670977s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.143987792 (8h16m40.733332779s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.145683708 (8h16m40.735028695s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.179729887 (8h16m40.769074874s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.184258271 (8h16m40.773603258s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189182012 (8h16m40.778526989s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189550136 (8h16m40.778895123s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.190054842 (8h16m40.779399829s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.197377522 (8h16m40.786722509s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.203901384 (8h16m40.793246411s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.212457535 (8h16m40.801802522s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218587324 (8h16m40.807932311s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218619645 (8h16m40.807964662s) [seg_pos={0 0}, upload=8641]: Segment reached optimal threshold of 14 pieces. Aug 7 17:56:41.218795952 (8h16m40.808140939s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1251oUFjESKjnTubb2wF8K9nsa8fztBm4nup3GvAymHLExfH35v): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218830829 (8h16m40.808175806s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ec4Xvp3nGRAm9BivLLTp6AVkMoz1i2AgXQdfQaAmNPLHWD4Yv): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218923186 (8h16m40.808268233s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gJKrCBLJWN1SBKibfeqBtcqqiv4veKTGmtnycjZyb3VN9kwoT): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218971639 (8h16m40.808316626s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b1UCF4dQobQEVqe9XB5tKxDx9L2P7kkQW179fxc1ir9ZtQVsM): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218999302 (8h16m40.808344279s) [seg_pos={0 0}, upload=8641]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:56:41.219090676 (8h16m40.808435663s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:56:41.219094864 (8h16m40.808439851s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:41.533326972 (8h16m41.122671949s) [upload=8641]: Done uploading segments... Aug 7 17:56:41.533346088 (8h16m41.122691065s) [upload=8641]: Upload finished. err: <nil> Duration: 6.313591975s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.859609096s FAIL (2025-08-07 17:59:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.859630185s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................X............................XXX.....] duration: 5.933747032s FAIL (2025-08-07 17:59:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== load=8629]: Done uploading segments... Aug 7 17:56:00.782013583 (8h16m0.37135857s) [upload=8629]: Upload finished. err: <nil> Aug 7 17:56:38.723311581 (8h16m38.312656568s) [upload=8641]: Starting upload Aug 7 17:56:38.723381295 (8h16m38.312726282s) [upload=8641]: Done starting upload Aug 7 17:56:38.723510432 (8h16m38.312855419s) [upload=8641]: Uploading segments... Aug 7 17:56:38.723689655 (8h16m38.313034642s) [upload=8641]: Got next segment. Inline: false Aug 7 17:56:38.723742636 (8h16m38.313087624s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:38.723747806 (8h16m38.313092813s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:56:39.185110639 (8h16m38.774455626s) [seg_pos={0 0}, upload=8641]: Begin upload segment... Aug 7 17:56:39.185155283 (8h16m38.77450027s) [seg_pos={0 0}, upload=8641]: Done begin upload segment. Aug 7 17:56:39.185179680 (8h16m38.774524657s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185283429 (8h16m38.774628416s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185591228 (8h16m38.774936215s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185791812 (8h16m38.775136809s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185997285 (8h16m38.775342272s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186141400 (8h16m38.775486387s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186247333 (8h16m38.77559232s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186429762 (8h16m38.775774739s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186588346 (8h16m38.775933333s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186724576 (8h16m38.776069563s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186842933 (8h16m38.77618791s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187031925 (8h16m38.776376912s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187135412 (8h16m38.776480399s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187272524 (8h16m38.776617511s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187389569 (8h16m38.776734546s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187506292 (8h16m38.776851279s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187618787 (8h16m38.776963774s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187887912 (8h16m38.777232889s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:40.297071894 (8h16m39.886416881s) [upload=8641]: Next returned nil segment Aug 7 17:56:40.297094217 (8h16m39.886439204s) [upload=8641]: Waiting for error group managing segments... Aug 7 17:56:40.607636446 (8h16m40.196981433s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.119004102 (8h16m40.708349089s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.138326000 (8h16m40.727670977s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.143987792 (8h16m40.733332779s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.145683708 (8h16m40.735028695s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.179729887 (8h16m40.769074874s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.184258271 (8h16m40.773603258s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189182012 (8h16m40.778526989s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189550136 (8h16m40.778895123s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.190054842 (8h16m40.779399829s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.197377522 (8h16m40.786722509s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.203901384 (8h16m40.793246411s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.212457535 (8h16m40.801802522s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218587324 (8h16m40.807932311s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218619645 (8h16m40.807964662s) [seg_pos={0 0}, upload=8641]: Segment reached optimal threshold of 14 pieces. Aug 7 17:56:41.218795952 (8h16m40.808140939s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1251oUFjESKjnTubb2wF8K9nsa8fztBm4nup3GvAymHLExfH35v): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218830829 (8h16m40.808175806s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ec4Xvp3nGRAm9BivLLTp6AVkMoz1i2AgXQdfQaAmNPLHWD4Yv): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218923186 (8h16m40.808268233s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gJKrCBLJWN1SBKibfeqBtcqqiv4veKTGmtnycjZyb3VN9kwoT): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218971639 (8h16m40.808316626s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b1UCF4dQobQEVqe9XB5tKxDx9L2P7kkQW179fxc1ir9ZtQVsM): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218999302 (8h16m40.808344279s) [seg_pos={0 0}, upload=8641]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:56:41.219090676 (8h16m40.808435663s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:56:41.219094864 (8h16m40.808439851s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:41.533326972 (8h16m41.122671949s) [upload=8641]: Done uploading segments... Aug 7 17:56:41.533346088 (8h16m41.122691065s) [upload=8641]: Upload finished. err: <nil> Duration: 5.933770507s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.841107016s FAIL (2025-08-07 17:59:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 97d8163ae14062859bb5f577475cb6368637e6fc361167939fd0819511d5eec6 != before eaae4c283ed51a819191bed42482bec8abad60e74bfff9dc02b59186162b78d6 ======LOGS====== em: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:51:44.777575282 (8h11m44.366920269s) [upload=8556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:51:45.128715205 (8h11m44.718060192s) [upload=8556]: Done uploading segments... Aug 7 17:51:45.128733039 (8h11m44.718078026s) [upload=8556]: Upload finished. err: <nil> Aug 7 17:52:13.419119646 (8h12m13.008464633s) [upload=8565]: Starting upload Aug 7 17:52:13.419153872 (8h12m13.008498849s) [upload=8565]: Done starting upload Aug 7 17:52:13.419226821 (8h12m13.008571808s) [upload=8565]: Uploading segments... Aug 7 17:52:13.419263201 (8h12m13.008608188s) [upload=8565]: Got next segment. Inline: true Aug 7 17:52:13.419267669 (8h12m13.008612656s) [upload=8565]: Waiting for error group managing segments... Aug 7 17:52:13.419328786 (8h12m13.008673783s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:13.419333575 (8h12m13.008678562s) [upload=8565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:13.419335859 (8h12m13.008680846s) [upload=8565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:13.723776093 (8h12m13.31312108s) [upload=8565]: Done uploading segments... Aug 7 17:52:13.723799456 (8h12m13.313144444s) [upload=8565]: Upload finished. err: <nil> Aug 7 17:52:47.034488259 (8h12m46.623833246s) [upload=8577]: Starting upload Aug 7 17:52:47.034530921 (8h12m46.623875898s) [upload=8577]: Done starting upload Aug 7 17:52:47.034571168 (8h12m46.623916155s) [upload=8577]: Uploading segments... Aug 7 17:52:47.034599672 (8h12m46.623944659s) [upload=8577]: Got next segment. Inline: true Aug 7 17:52:47.034603349 (8h12m46.623948336s) [upload=8577]: Waiting for error group managing segments... Aug 7 17:52:47.034633136 (8h12m46.623978123s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:52:47.034636602 (8h12m46.62398159s) [upload=8577]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:52:47.034638827 (8h12m46.623983804s) [upload=8577]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:52:47.338056735 (8h12m46.927401712s) [upload=8577]: Done uploading segments... Aug 7 17:52:47.338073597 (8h12m46.927418584s) [upload=8577]: Upload finished. err: <nil> Aug 7 17:53:57.176128309 (8h13m56.765473296s) [upload=8596]: Starting upload Aug 7 17:53:57.176163446 (8h13m56.765508433s) [upload=8596]: Done starting upload Aug 7 17:53:57.176210717 (8h13m56.765555694s) [upload=8596]: Uploading segments... Aug 7 17:53:57.176240193 (8h13m56.76558517s) [upload=8596]: Got next segment. Inline: true Aug 7 17:53:57.176244351 (8h13m56.765589338s) [upload=8596]: Waiting for error group managing segments... Aug 7 17:53:57.176282755 (8h13m56.765627752s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:53:57.176288726 (8h13m56.765633743s) [upload=8596]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:53:57.176293325 (8h13m56.765638312s) [upload=8596]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:53:57.449047494 (8h13m57.038392481s) [upload=8596]: Done uploading segments... Aug 7 17:53:57.449071139 (8h13m57.038416126s) [upload=8596]: Upload finished. err: <nil> Aug 7 17:54:49.189941218 (8h14m48.779286195s) [upload=8611]: Starting upload Aug 7 17:54:49.189993819 (8h14m48.779338806s) [upload=8611]: Done starting upload Aug 7 17:54:49.190040198 (8h14m48.779385185s) [upload=8611]: Uploading segments... Aug 7 17:54:49.190072840 (8h14m48.779417817s) [upload=8611]: Got next segment. Inline: true Aug 7 17:54:49.190078170 (8h14m48.779423157s) [upload=8611]: Waiting for error group managing segments... Aug 7 17:54:49.190110843 (8h14m48.77945583s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:54:49.190114651 (8h14m48.779459638s) [upload=8611]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:54:49.190116875 (8h14m48.779461862s) [upload=8611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:54:49.635851345 (8h14m49.225196332s) [upload=8611]: Done uploading segments... Aug 7 17:54:49.635866053 (8h14m49.22521104s) [upload=8611]: Upload finished. err: <nil> Aug 7 17:55:36.545614229 (8h15m36.134959216s) [upload=8624]: Starting upload Aug 7 17:55:36.545646951 (8h15m36.134991938s) [upload=8624]: Done starting upload Aug 7 17:55:36.545692439 (8h15m36.135037426s) [upload=8624]: Uploading segments... Aug 7 17:55:36.545719010 (8h15m36.135063987s) [upload=8624]: Got next segment. Inline: true Aug 7 17:55:36.545728027 (8h15m36.135073014s) [upload=8624]: Waiting for error group managing segments... Aug 7 17:55:36.545759767 (8h15m36.135104754s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:55:36.545763394 (8h15m36.135108371s) [upload=8624]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:55:36.545765829 (8h15m36.135110817s) [upload=8624]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:55:36.818640298 (8h15m36.407985285s) [upload=8624]: Done uploading segments... Aug 7 17:55:36.818658132 (8h15m36.408003119s) [upload=8624]: Upload finished. err: <nil> Aug 7 17:56:06.578129577 (8h16m6.167474584s) [upload=8631]: Starting upload Aug 7 17:56:06.578220451 (8h16m6.167565458s) [upload=8631]: Done starting upload Aug 7 17:56:06.578296086 (8h16m6.167641093s) [upload=8631]: Uploading segments... Aug 7 17:56:06.578337214 (8h16m6.167682201s) [upload=8631]: Got next segment. Inline: true Aug 7 17:56:06.578342465 (8h16m6.167687462s) [upload=8631]: Waiting for error group managing segments... Aug 7 17:56:06.578387570 (8h16m6.167732567s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:06.578392750 (8h16m6.167737737s) [upload=8631]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:56:06.578396137 (8h16m6.167741134s) [upload=8631]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:06.849533802 (8h16m6.438878779s) [upload=8631]: Done uploading segments... Aug 7 17:56:06.849551326 (8h16m6.438896313s) [upload=8631]: Upload finished. err: <nil> Aug 7 17:57:16.502605544 (8h17m16.091950531s) [upload=8654]: Starting upload Aug 7 17:57:16.502657013 (8h17m16.092002s) [upload=8654]: Done starting upload Aug 7 17:57:16.502702439 (8h17m16.092047416s) [upload=8654]: Uploading segments... Aug 7 17:57:16.502730643 (8h17m16.09207562s) [upload=8654]: Got next segment. Inline: true Aug 7 17:57:16.502734391 (8h17m16.092079378s) [upload=8654]: Waiting for error group managing segments... Aug 7 17:57:16.502821517 (8h17m16.092166504s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:16.502826106 (8h17m16.092171083s) [upload=8654]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:16.502828360 (8h17m16.092173347s) [upload=8654]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:16.814985979 (8h17m16.404330966s) [upload=8654]: Done uploading segments... Aug 7 17:57:16.815003262 (8h17m16.404348249s) [upload=8654]: Upload finished. err: <nil> Aug 7 17:57:49.828006242 (8h17m49.417351219s) [upload=8662]: Starting upload Aug 7 17:57:49.828057911 (8h17m49.417402888s) [upload=8662]: Done starting upload Aug 7 17:57:49.828138335 (8h17m49.417483322s) [upload=8662]: Uploading segments... Aug 7 17:57:49.828231282 (8h17m49.417576269s) [upload=8662]: Got next segment. Inline: true Aug 7 17:57:49.828236323 (8h17m49.41758131s) [upload=8662]: Waiting for error group managing segments... Aug 7 17:57:49.828292991 (8h17m49.417637968s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:57:49.828298201 (8h17m49.417643188s) [upload=8662]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:57:49.828300595 (8h17m49.417645572s) [upload=8662]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:57:50.093437280 (8h17m49.682782267s) [upload=8662]: Done uploading segments... Aug 7 17:57:50.093454993 (8h17m49.68279998s) [upload=8662]: Upload finished. err: <nil> Aug 7 17:58:46.298822170 (8h18m45.888167157s) [upload=8680]: Starting upload Aug 7 17:58:46.298867998 (8h18m45.888212975s) [upload=8680]: Done starting upload Aug 7 17:58:46.298938203 (8h18m45.88828319s) [upload=8680]: Uploading segments... Aug 7 17:58:46.299038384 (8h18m45.888383371s) [upload=8680]: Got next segment. Inline: true Aug 7 17:58:46.299043324 (8h18m45.888388311s) [upload=8680]: Waiting for error group managing segments... Aug 7 17:58:46.299100473 (8h18m45.88844545s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:58:46.299105533 (8h18m45.88845052s) [upload=8680]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:58:46.299107707 (8h18m45.888452694s) [upload=8680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:58:46.576925564 (8h18m46.166270551s) [upload=8680]: Done uploading segments... Aug 7 17:58:46.576940522 (8h18m46.166285509s) [upload=8680]: Upload finished. err: <nil> Aug 7 17:59:04.060415740 (8h19m3.649760727s) [upload=8684]: Starting upload Aug 7 17:59:04.060460084 (8h19m3.649805062s) [upload=8684]: Done starting upload Aug 7 17:59:04.060532463 (8h19m3.64987745s) [upload=8684]: Uploading segments... Aug 7 17:59:04.060685006 (8h19m3.650030023s) [upload=8684]: Got next segment. Inline: true Aug 7 17:59:04.060693351 (8h19m3.650038338s) [upload=8684]: Waiting for error group managing segments... Aug 7 17:59:04.060754989 (8h19m3.650099976s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:59:04.060760259 (8h19m3.650105246s) [upload=8684]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 17:59:04.060762493 (8h19m3.65010748s) [upload=8684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:59:04.338666545 (8h19m3.928011532s) [upload=8684]: Done uploading segments... Aug 7 17:59:04.338683498 (8h19m3.928028475s) [upload=8684]: Upload finished. err: <nil> Duration: 1.841130802s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.......................X............................XXX....] duration: 5.971473763s FAIL (2025-08-07 17:59:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== load=8629]: Done uploading segments... Aug 7 17:56:00.782013583 (8h16m0.37135857s) [upload=8629]: Upload finished. err: <nil> Aug 7 17:56:38.723311581 (8h16m38.312656568s) [upload=8641]: Starting upload Aug 7 17:56:38.723381295 (8h16m38.312726282s) [upload=8641]: Done starting upload Aug 7 17:56:38.723510432 (8h16m38.312855419s) [upload=8641]: Uploading segments... Aug 7 17:56:38.723689655 (8h16m38.313034642s) [upload=8641]: Got next segment. Inline: false Aug 7 17:56:38.723742636 (8h16m38.313087624s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 17:56:38.723747806 (8h16m38.313092813s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 17:56:39.185110639 (8h16m38.774455626s) [seg_pos={0 0}, upload=8641]: Begin upload segment... Aug 7 17:56:39.185155283 (8h16m38.77450027s) [seg_pos={0 0}, upload=8641]: Done begin upload segment. Aug 7 17:56:39.185179680 (8h16m38.774524657s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185283429 (8h16m38.774628416s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185591228 (8h16m38.774936215s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185791812 (8h16m38.775136809s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.185997285 (8h16m38.775342272s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186141400 (8h16m38.775486387s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186247333 (8h16m38.77559232s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186429762 (8h16m38.775774739s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186588346 (8h16m38.775933333s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186724576 (8h16m38.776069563s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.186842933 (8h16m38.77618791s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187031925 (8h16m38.776376912s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187135412 (8h16m38.776480399s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187272524 (8h16m38.776617511s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187389569 (8h16m38.776734546s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187506292 (8h16m38.776851279s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187618787 (8h16m38.776963774s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:39.187887912 (8h16m38.777232889s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Uploading piece... Aug 7 17:56:40.297071894 (8h16m39.886416881s) [upload=8641]: Next returned nil segment Aug 7 17:56:40.297094217 (8h16m39.886439204s) [upload=8641]: Waiting for error group managing segments... Aug 7 17:56:40.607636446 (8h16m40.196981433s) [noise=true , address=79.127.205.245:4229 , piece_id=FPOCYO7B3NRPBBKZ54WLNQJVAOMHBH24MYMJVH3TWWFPVTTRRWIA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.119004102 (8h16m40.708349089s) [noise=true , address=104.36.82.58:4275 , piece_id=JCNGRMCHOGENUWGSZSUOFIHFXHORY6GOADZ3BYE2WWPOVXBRZGKA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.138326000 (8h16m40.727670977s) [noise=true , address=104.36.82.55:4221 , piece_id=34OYJQC6A6A2DYJXHNWTRUMSGQZOAWRWWRIQFWQLCKY7L2CIMC7A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.143987792 (8h16m40.733332779s) [noise=true , address=45.91.116.7:4020 , piece_id=J6MHIRA6BZYCJJA7QPZ4ND5YX76ZTRTM73IBJPI7NEAAWBDVHFGA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.145683708 (8h16m40.735028695s) [noise=true , address=104.36.82.53:4154 , piece_id=NTYF7MFADJWZAIJ2K6SMBDAOKMC7Z7EMVN7BQTEYXIZ4PURB2BXQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.179729887 (8h16m40.769074874s) [noise=true , address=104.36.82.45:4930 , piece_id=DGVSF7DTI2TJ6WP5WBHJZ4UIRFSPZCFGSQJZQZOBSFPUOT4G4L5A, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.184258271 (8h16m40.773603258s) [noise=true , address=104.36.82.22:4543 , piece_id=FP3DGUA26BAKEUXBX6QKTWVRD5M33ESSM2SPBSHHSK54RUVAJLYQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189182012 (8h16m40.778526989s) [noise=true , address=156.146.43.225:4462 , piece_id=YOVL33D5MCGTQAWYFZZFUNIONHC2XWJSCCUJID3EBNNJYP4VE2GA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.189550136 (8h16m40.778895123s) [noise=true , address=104.36.82.8:4990 , piece_id=NKL7RUWHQVQQPDIEJP3HBZ2JAM2SGY2CLUFBBSN6ZKHI7CFAOVRQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.190054842 (8h16m40.779399829s) [noise=true , address=104.36.82.61:4395 , piece_id=OYGO7WXQYGVJ42YW26AF5YMQUDPVM36KATLG4YX345FHODN5O3SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.197377522 (8h16m40.786722509s) [noise=true , address=104.36.82.54:4192 , piece_id=D2ZGCZRWHTA3BCVKD5WSUTKEOS2JCZACFI4EDGI7U7IKU4XVY2UA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.203901384 (8h16m40.793246411s) [noise=true , address=104.36.82.47:4541 , piece_id=RQDTFLWULHTHM42J6ZYT632C5WRXPURS7ODS3ATE3JWVXZFGV4SA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.212457535 (8h16m40.801802522s) [noise=true , address=104.36.82.46:4514 , piece_id=4J2H54R3BI4VWMYGDDEW2GCUVGUC7AV5PVHR2DPA4CD2JHU242FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218587324 (8h16m40.807932311s) [noise=true , address=104.36.82.9:4019 , piece_id=KQ3HGBXGWJPCPIFOEW6PSJXLPO6QUHWB3YQT55WPRTSTQCMVAQWA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: <nil> Aug 7 17:56:41.218619645 (8h16m40.807964662s) [seg_pos={0 0}, upload=8641]: Segment reached optimal threshold of 14 pieces. Aug 7 17:56:41.218795952 (8h16m40.808140939s) [noise=true , address=104.36.82.4:4831 , piece_id=HEF7BKSMBVUZJ42RXKIOYLL6JL27GD4BY2SB3BJHMNQ64B3ICEZQ, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1251oUFjESKjnTubb2wF8K9nsa8fztBm4nup3GvAymHLExfH35v): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218830829 (8h16m40.808175806s) [noise=true , address=104.36.82.16:4303 , piece_id=3WSSGY4HP5JZULTVKKSUD25R3ODRIXZRCCRKF4SQWBQ74ZYD35FA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ec4Xvp3nGRAm9BivLLTp6AVkMoz1i2AgXQdfQaAmNPLHWD4Yv): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218923186 (8h16m40.808268233s) [noise=true , address=104.36.82.59:4303 , piece_id=RUE7IUX5WHG4W3GRSQ536EDTI2GP4YOMVXVQ2FSHITTXZT7G6LLA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gJKrCBLJWN1SBKibfeqBtcqqiv4veKTGmtnycjZyb3VN9kwoT): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218971639 (8h16m40.808316626s) [noise=true , address=104.36.82.37:4669 , piece_id=4XFIQAKDRXMKBGLTHNLG6WUP2R2G4WK5ZV2EL6X6IZGXCRVMRIPA, seg_pos={0 0}, upload=8641]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b1UCF4dQobQEVqe9XB5tKxDx9L2P7kkQW179fxc1ir9ZtQVsM): protocol: expected piece hash; context canceled; EOF Aug 7 17:56:41.218999302 (8h16m40.808344279s) [seg_pos={0 0}, upload=8641]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 17:56:41.219090676 (8h16m40.808435663s) [upload=8641]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 17:56:41.219094864 (8h16m40.808439851s) [upload=8641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 17:56:41.533326972 (8h16m41.122671949s) [upload=8641]: Done uploading segments... Aug 7 17:56:41.533346088 (8h16m41.122691065s) [upload=8641]: Upload finished. err: <nil> Duration: 5.971490916s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)