Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.452893017s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:33:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.452942031s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
3.391721492s trace:
https://jaeger.dev.storj.io/trace/7173e01acd2fefef
FAIL (2025-11-04 09:32:49 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.39173597s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.807764466s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:32:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.807788944s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.794953025s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:31:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.794981089s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001735895s trace:
https://jaeger.dev.storj.io/trace/7173e01acd2fefef
FAIL (2025-11-04 09:31:40 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001761935s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.838206158s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:31:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.838238018s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.424357035s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:30:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.424385679s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.881650433s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:29:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.881667366s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.469968s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:29:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.469989071s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
3.407426499s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:28:44 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.40745275s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
3.40417081s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:28:28 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.404199696s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.779060181s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:28:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.779093174s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
3.383948481s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:28:11 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.383979632s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
3.411429098s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:27:13 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.411455199s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.360835861s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:27:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.360875385s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
3.380205254s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:26:50 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.380238397s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
5.90750888s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:26:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.907526074s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m46.283045405s trace:
https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5
FAIL (2025-11-04 09:26:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece...
Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment
Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments...
Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil>
Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces.
Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF
Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold
Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments...
Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m46.283060143s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
3.397833522s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:26:20 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.397866405s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
3.459729914s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:25:18 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.459762076s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
3.344507029s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:25:02 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.344535924s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
3.358607018s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:24:35 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.358668224s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
3.380963309s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:23:24 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.380991352s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
3.350487606s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:22:15 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.35051665s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
3.351120102s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:21:39 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.351146301s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............X...............................................] duration:
3.412997081s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:21:13 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.41303914s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............X..............................................] duration:
3.44081364s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:20:48 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.440834419s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............X.............................................] duration:
3.394052763s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:20:11 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.3940824s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............X............................................] duration:
3.451464674s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:19:53 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.45149419s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................X...........................................] duration:
3.397144242s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:19:13 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.397162015s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................X..........................................] duration:
3.412234404s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:18:43 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.412252427s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................X.........................................] duration:
3.361994316s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:18:07 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.362023051s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................X........................................] duration:
3.401889435s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:17:07 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.401917839s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................X.......................................] duration:
3.421487496s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:16:31 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.421547962s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................X......................................] duration:
3.379294348s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:15:41 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.379320841s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................X.....................................] duration:
3.365516776s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:15:26 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.365594184s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.931977568s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:14:43 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 1.932019468s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................X....................................] duration:
3.393838253s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:14:18 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.393868982s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.98034082s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:13:29 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 1.980364244s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................X...................................] duration:
3.360463178s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:13:29 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.360500509s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.966768346s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:13:02 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 1.96679687s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
3.379019624s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:12:29 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.379051014s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.072215419s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:12:25 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.072235737s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
3.399244608s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:11:39 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.399274564s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.031411382s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:11:34 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.031529447s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................X................................] duration:
3.419515226s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:11:07 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.41954897s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.018632496s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:10:56 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.018665509s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................X...............................] duration:
3.390067468s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:10:28 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.390205822s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................X..............................] duration:
3.487012524s trace:
https://jaeger.dev.storj.io/trace/6b308ae490519c20
FAIL (2025-11-04 09:10:05 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.487047871s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.992249341s trace:
https://jaeger.dev.storj.io/trace/1d7439731666bbaf
FAIL (2025-11-04 09:09:44 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 1.992283415s
core
uplink
us1
|