Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 3.788263546s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:47:21 UTC) orbiter warning: context deadline exceeded Duration: 3.788278855s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 3.686422042s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:47:05 UTC) orbiter warning: context deadline exceeded Duration: 3.686508617s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 4.588616783s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:46:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.588644376s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 4.71639014s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:46:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.71641609s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [..X.X..........X................X........X....XX.........X.X] duration: 1m0.000519302s trace: https://jaeger.dev.storj.io/trace/155d90e3cdd65b60 FAIL (2026-05-17 22:46:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000543087s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [X...X.......................................................] duration: 3.89228913s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:46:10 UTC) orbiter warning: context deadline exceeded Duration: 3.892306723s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 4.592884998s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:45:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.592910218s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.X...X......................................................] duration: 3.792565414s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:45:34 UTC) orbiter warning: context deadline exceeded Duration: 3.79258963s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..X...X.....................................................] duration: 3.659537511s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:45:07 UTC) orbiter warning: context deadline exceeded Duration: 3.659564533s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 4.708378813s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:45:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.708400875s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.00715531s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:44:38 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 5.00718199s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [...X.X..........X................X........X....XX.........X.] duration: 6.63861071s trace: https://jaeger.dev.storj.io/trace/69973142caecfc29 FAIL (2026-05-17 22:44:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.638627471s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [....X.X..........X................X........X....XX.........X] duration: 1m0.000952892s trace: https://jaeger.dev.storj.io/trace/69973142caecfc29 FAIL (2026-05-17 22:44:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000989892s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 4.570375377s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:44:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.570403149s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...X...X....................................................] duration: 3.579383782s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:44:00 UTC) orbiter warning: context deadline exceeded Duration: 3.579406615s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 4.291152871s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:43:30 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.291181125s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....X...X...................................................] duration: 3.721002579s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:43:12 UTC) orbiter warning: context deadline exceeded Duration: 3.721023288s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 4.697282729s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:42:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.697308239s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.....X.X..........X................X........X....XX.........] duration: 9.743866379s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:42:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.74392433s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....X...X..................................................] duration: 3.870762842s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:42:31 UTC) orbiter warning: context deadline exceeded Duration: 3.8707831s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 4.678276483s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:42:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.678297973s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......X...X.................................................] duration: 4.616026658s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:41:57 UTC) orbiter warning: context deadline exceeded Duration: 4.616233663s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [......X.X..........X................X........X....XX........] duration: 7.845760496s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:41:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.845786385s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 4.604406961s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:41:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.604427491s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.......X.X..........X................X........X....XX.......] duration: 46.330556811s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:41:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 46.330581126s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [X......X...X................................................] duration: 5.001092042s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:40:52 UTC) orbiter warning: context deadline exceeded Duration: 5.001124054s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 4.25944655s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:40:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.259472799s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 4.98020874s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:39:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.98024002s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [........X.X..........X................X........X....XX......] duration: 18.017245923s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:39:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.017268366s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.X......X...X...............................................] duration: 3.775047974s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:39:38 UTC) orbiter warning: context deadline exceeded Duration: 3.775086278s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.........X.X..........X................X........X....XX.....] duration: 6.519347372s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:39:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.519366156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.130205148s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:39:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 5.13023791s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..X......X...X..............................................] duration: 3.71212299s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:38:31 UTC) orbiter warning: context deadline exceeded Duration: 3.712167275s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [..........X.X..........X................X........X....XX....] duration: 8.602383487s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:38:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.602405729s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...X......X...X.............................................] duration: 3.737119963s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:37:59 UTC) orbiter warning: context deadline exceeded Duration: 3.737149841s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.100541362s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:37:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 5.100567622s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....X......X...X............................................] duration: 4.438927088s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:37:40 UTC) orbiter warning: context deadline exceeded Duration: 4.438946174s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [...........X.X..........X................X........X....XX...] duration: 7.139570592s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:37:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.139881235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....X......X...X...........................................] duration: 3.757158754s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:37:16 UTC) orbiter warning: context deadline exceeded Duration: 3.757190836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [............X.X..........X................X........X....XX..] duration: 33.308867417s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:37:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 33.308891101s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.094449547s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:36:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 5.094467172s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [......X......X...X..........................................] duration: 4.943620847s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:36:50 UTC) orbiter warning: context deadline exceeded Duration: 4.943643681s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.......X......X...X.........................................] duration: 4.078870599s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:36:19 UTC) orbiter warning: context deadline exceeded Duration: 4.078893663s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.............X.X..........X................X........X....XX.] duration: 7.097770571s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:36:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.097785119s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.606704103s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:36:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 4.606730523s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [..............X.X..........X................X........X....XX] duration: 1m0.001321388s trace: https://jaeger.dev.storj.io/trace/1320c8183ac5e9a1 FAIL (2026-05-17 22:36:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001331217s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [........X......X...X........................................] duration: 3.688513609s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:35:50 UTC) orbiter warning: context deadline exceeded Duration: 3.688555339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.509972711s trace: https://jaeger.dev.storj.io/trace/76f682843bbf000c FAIL (2026-05-17 22:35:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== , upload=394782]: Uploading piece... May 17 22:34:27.036102035 (411h46m37.55692578s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.036587391 (411h46m37.557411136s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037078427 (411h46m37.557902172s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.037484623 (411h46m37.558308368s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.038060963 (411h46m37.558884708s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.039286852 (411h46m37.560110597s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.040726409 (411h46m37.561550154s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.041849241 (411h46m37.562672986s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.043210408 (411h46m37.564034153s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.044433683 (411h46m37.565257428s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Uploading piece... May 17 22:34:27.854914283 (411h46m38.375738028s) [upload=394782]: Next returned nil segment May 17 22:34:27.854924823 (411h46m38.375748568s) [upload=394782]: Waiting for error group managing segments... May 17 22:34:28.239545317 (411h46m38.760369062s) [noise=true , address=185.167.204.205:4009 , piece_id=LO7H4S3JKJKRSOGJ6ZODPFJ6YBTUU2LDUY3CTLPY23X6S57HZN3Q, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.245786637 (411h46m38.766610382s) [noise=true , address=185.167.204.206:4068 , piece_id=IZTG3A67SWKZX5C526KTTEAOISUMKCHIFLWL3MTPDSI6QI7CP65A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.255840251 (411h46m38.776663996s) [noise=true , address=185.167.204.206:4059 , piece_id=IM5N2WKCNHD6M57X5VR6TIJE4F6IZEDEP4AM4FQ3KWYG4STVEOGQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.258630967 (411h46m38.779454712s) [noise=true , address=185.167.204.206:4060 , piece_id=AH64NEU2RFVYZAICJSRWBU3OASGCKX2OZLNBPGYR3AJKUG4K7JWA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260445739 (411h46m38.781269484s) [noise=true , address=185.167.204.206:4057 , piece_id=GNJGLIDMRVKVPTBPAL5YDCM37ZA6SQBMRWFLZLRIEWRXAI7GRKFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.260820615 (411h46m38.78164436s) [noise=true , address=185.167.204.205:4034 , piece_id=ALXLNAFX7EQQEKS3XIVXKLDONO64CYM6GUFGNGY5OS7SSYECKHZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.261604741 (411h46m38.782428486s) [noise=true , address=185.167.204.205:4032 , piece_id=VWRM2HTEQY6HZ2XG3267SEFYFTNWABAJIBVULFUWJBQA4AEPGRFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.266260546 (411h46m38.787084281s) [noise=true , address=185.167.204.205:4029 , piece_id=SVSUZ27ZU3FNO4L4WSLHIUXNW6JOPINOKXNAI2IWDYTMAZKEZZMA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.287363153 (411h46m38.808186898s) [noise=true , address=81.171.14.70:4041 , piece_id=SNOP2G7CBJBICB56L43KJJ52JHNJHFDNHAHVCDDHHFPX7NPZ6SFQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.301235895 (411h46m38.82205965s) [noise=true , address=81.171.14.70:4043 , piece_id=LEVZBMTS27YEYUY3GWP5C6EFKXJ4BP6XF6YR6VRYHNFGIJGRQADQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.307957311 (411h46m38.828781056s) [noise=true , address=81.171.14.70:4033 , piece_id=DDKWWNUZS2RWWRH2KDLVCBWAPSQ5EALFEWGGDYPSQM74TF7MYA5A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.308575370 (411h46m38.829399135s) [noise=true , address=81.171.14.70:4045 , piece_id=UNQMYVTTY2GZ3QOZOYF3AEGUS2KSAZNTXVYM2RNXMZ74E6CVULRQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.312344021 (411h46m38.833167766s) [noise=true , address=81.171.14.70:4044 , piece_id=CUABE6O2PEW7L6ZAL5HGPBMTRZAZJ4XR67MH2KKAKAFAW6X2ERHA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323450836 (411h46m38.844274571s) [noise=true , address=81.171.14.70:4035 , piece_id=V6P5EVDK3GAKZ6KFPAPUJC6UVCBW2XNYZSTLXOF7P3WFSKKQP4FQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.323838546 (411h46m38.844662291s) [noise=true , address=81.171.14.70:4025 , piece_id=Z4ETZSWLMFIV4LHTKVKJSAJI5OBUTUZRCDZWDEQXZ4YJRHDNH5PA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.330862219 (411h46m38.851685964s) [noise=true , address=81.171.14.70:4029 , piece_id=JKORZSUDHGDR6IKR6HEWFUMWESJ7C5F7WPOSAHPDSNMTFN6P4OZQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331259857 (411h46m38.852083602s) [noise=true , address=81.171.14.70:4034 , piece_id=PUSHGLRUZN2GIIIRSIZBZC6EJXIP2PXA5XZXBAVVTD4CZXXCW5CA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.331595307 (411h46m38.852419072s) [noise=true , address=81.171.14.70:4038 , piece_id=AEAARROU4DSXBAQWBJYWPYGR6T7K6SXESNVPNQPBK4PTPZ6FCXBQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.338839270 (411h46m38.859663015s) [noise=true , address=81.171.14.70:4046 , piece_id=3KFORIUT4DE5I7DN3NOHNSMLKIMSVDZQPTUY6MOSENHILE2NZS7A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.794664577 (411h46m39.315488322s) [noise=true , address=89.149.201.147:4020 , piece_id=F7FUUHN7QF4YYJXB6U3IFARFXAITRUGW3BOBKPBJXOXFG77KQM4A, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.855136180 (411h46m39.375959925s) [noise=true , address=89.149.201.147:4008 , piece_id=3LRTQX5GV6PK27NGLOGX3E6EP5WEABOJS3NZMRUG6ZAYFGDPCJAA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.881838443 (411h46m39.402662188s) [noise=true , address=89.149.201.147:4021 , piece_id=DUSL2NLVZOBEXT56ELQCRQXZZHU4TA3AJOD5MEPDFVDXXMDEDNXA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.883663846 (411h46m39.404487591s) [noise=true , address=89.149.201.147:4009 , piece_id=JPNCTOQ3PTXFR4OFZ636GN72XJC3GFBMZ3KKNIZRPRXYCXGV36AA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:28.926714846 (411h46m39.447538591s) [noise=true , address=89.149.201.147:4024 , piece_id=27QK7WKX4UZVNOYVAHSIDZLKEBL3KWAPOFS6MRXBFF3LNNMS3UYA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.213587658 (411h46m39.734411403s) [noise=true , address=89.149.201.147:4003 , piece_id=GQSH37SWTS64URW6DWSJMHMPX3P64Q7ZBQSAQRYPWX5BLBZYFCTA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380200989 (411h46m39.901024734s) [noise=true , address=89.149.201.147:4019 , piece_id=SH4DF4Q5WQCOJXML2EUKM5WN64PGXRW6VJ4QBQUZI2CTMMBY5FWQ, seg_pos={0 0}, upload=394782]: Done uploading piece. err: <nil> May 17 22:34:29.380217460 (411h46m39.901041205s) [seg_pos={0 0}, upload=394782]: Satisfied cohort requirements after 26 pieces May 17 22:34:29.380365422 (411h46m39.901189168s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380370573 (411h46m39.901194318s) [noise=true , address=89.149.201.147:4022 , piece_id=6KXNVI4Q5MYRWQUJIUVYNG5LOGEVYMG3YHGEWCO5HVFRLSHBLTDA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380539254 (411h46m39.901362999s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF May 17 22:34:29.380544274 (411h46m39.901368019s) [noise=true , address=89.149.201.147:4017 , piece_id=YE6BAX56XMAPCR52U5TSHT5Y3GD4DYNSB3JR6GNF4GZAK62VF6YA, seg_pos={0 0}, upload=394782]: Long tail cancellation due to optimal threshold May 17 22:34:29.380559463 (411h46m39.901383208s) [seg_pos={0 0}, upload=394782]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> May 17 22:34:29.380617765 (411h46m39.90144151s) [upload=394782]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 17 22:34:29.380621762 (411h46m39.901445497s) [upload=394782]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 17 22:34:29.628639019 (411h46m40.149462754s) [upload=394782]: Done uploading segments... May 17 22:34:29.628653306 (411h46m40.149477051s) [upload=394782]: Upload finished. err: <nil> Duration: 5.509997659s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...............X.X..........X................X........X....X] duration: 1m0.000966203s trace: https://jaeger.dev.storj.io/trace/4418067137bbe2f7 FAIL (2026-05-17 22:35:07 UTC) orbiter warning: context deadline exceeded Duration: 1m0.000976422s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........X......X...X.......................................] duration: 4.673547574s trace: https://jaeger.dev.storj.io/trace/65756aa66f3634b4 FAIL (2026-05-17 22:35:00 UTC) orbiter warning: context deadline exceeded Duration: 4.673572891s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)