Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.92006363s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:17:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.92009551s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.49423252s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:17:02 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.494401351s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.075947868s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:16:52 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.076045392s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 3.488331101s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:16:48 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.488360999s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.418914025s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:16:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.418946718s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.931535074s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:15:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.931626557s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 3.511982887s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:15:51 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.512011391s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.265678462s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:15:38 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.265702548s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.450315803s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:15:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.450366368s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.098752632s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:14:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 6.098770455s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 3.452125415s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:14:48 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.452209856s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.857932217s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:14:46 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.857963668s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.44504762s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:14:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.445061496s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 3.502042709s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:14:28 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.502137809s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.916288467s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:14:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.916348472s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.44643931s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:14:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.446468645s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 3.477289302s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:13:59 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.477325961s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.923364682s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:13:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.92338398s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.223095303s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:13:39 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.223126624s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.389633008s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:13:39 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.389659649s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.42094859s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:13:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.420981914s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 4.36712005s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:13:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.367150077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.02948859s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:12:47 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.029547343s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.993925951s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:12:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.993943323s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 3.41089767s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:12:39 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.410989546s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.388635248s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:12:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.388672128s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.890335681s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:11:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.890371378s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 3.535046866s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:11:40 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.535069309s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.372951633s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:11:36 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.372982553s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.405314319s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:11:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.405341441s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.15333745s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:11:09 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.153371374s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 3.545544295s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:11:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.545658903s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.407490563s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:10:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.407512985s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.182361942s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:10:42 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.182385425s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.039246632s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:10:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 6.039278082s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.356164687s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:10:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.356196518s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 3.572481352s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:10:03 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.572515007s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.878322891s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:09:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.878352617s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.731241319s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:09:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.731271226s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.428518191s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:09:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.428535293s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 3.514924469s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:09:28 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.514948004s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.430510275s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:08:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.430582473s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 3.499160841s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:08:47 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.499184156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.017189718s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:08:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 6.017217862s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 3.482784155s trace: https://jaeger.dev.storj.io/trace/251e9c48735b3b0a FAIL (2025-11-16 22:08:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.482827998s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.952177978s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:08:25 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.952210239s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.411521014s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:08:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.411545199s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.931294005s trace: https://jaeger.dev.storj.io/trace/6c6e7bbb319587a2 FAIL (2025-11-16 22:07:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Nov 16 21:49:44.857632528 (181h45m43.409102979s) [upload=160073]: Starting upload Nov 16 21:49:44.857711077 (181h45m43.409181538s) [upload=160073]: Done starting upload Nov 16 21:49:44.857867525 (181h45m43.409337986s) [upload=160073]: Uploading segments... Nov 16 21:49:44.858117080 (181h45m43.409587531s) [upload=160073]: Got next segment. Inline: false Nov 16 21:49:44.858184298 (181h45m43.409654749s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 16 21:49:44.858191282 (181h45m43.409661733s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 16 21:49:46.079436121 (181h45m44.630906582s) [seg_pos={0 0}, upload=160073]: Begin upload segment... Nov 16 21:49:46.079498500 (181h45m44.630968961s) [seg_pos={0 0}, upload=160073]: Done begin upload segment. Nov 16 21:49:46.079551421 (181h45m44.631021882s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079592329 (181h45m44.63106279s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.079721945 (181h45m44.631192406s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080339301 (181h45m44.631809762s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080638140 (181h45m44.632108601s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080980683 (181h45m44.632451144s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.080993116 (181h45m44.632463577s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081391766 (181h45m44.632862227s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.081400582 (181h45m44.632871043s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082534582 (181h45m44.634005043s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.082659149 (181h45m44.63412961s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083189538 (181h45m44.634659989s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.083640587 (181h45m44.635111048s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.084303179 (181h45m44.63577364s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.085550574 (181h45m44.637021025s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:46.091014480 (181h45m44.642484931s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Uploading piece... Nov 16 21:49:47.811332199 (181h45m46.36280266s) [upload=160073]: Next returned nil segment Nov 16 21:49:47.811354462 (181h45m46.362824913s) [upload=160073]: Waiting for error group managing segments... Nov 16 21:49:48.087848677 (181h45m46.639319128s) [noise=true , address=145.79.227.32:4834 , piece_id=4GON7GI63RXLBPJAJM32MO5BRMDTUFKJCRIJQDDWYDVOZA64LXGQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.105693638 (181h45m46.657164099s) [noise=true , address=145.79.227.25:4617 , piece_id=4H4BTA7PSPVFR7FJ7TCQCUOQPRVJCICXSBXUR6XJEZV5NSIBRZWQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106011864 (181h45m46.657482335s) [noise=true , address=145.79.227.10:4401 , piece_id=WIRM4I7DVYSHDQOEVCH23GRAC2G6HTQ2USSVL2HDTL6MO3VQQQJA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.106913180 (181h45m46.658383641s) [noise=true , address=145.79.227.15:4876 , piece_id=RSNHRRILCRJPBG6KJZTVFCQAROWHF5C6RBIB7XGSQH2SV3G6HUYQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.107526719 (181h45m46.65899718s) [noise=true , address=145.79.227.15:4873 , piece_id=OHIKS4CCPNMIVJAVNDZCPDXRL4PZLBLNMX6T6FV67N7S3BL75DZQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.114554281 (181h45m46.666024732s) [noise=true , address=145.79.227.30:4761 , piece_id=I256J6YH4LHPL4DU6H4P7NYFVJRI2XBV2AFJMASE2HPPDMO6QXLA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.119027128 (181h45m46.670497589s) [noise=true , address=145.79.227.11:4730 , piece_id=CPUKLG5IGGB3RLTVTPSXHCW6T25KJ3P7XNAMKOCRMOJ7UC4EVKDA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.133964262 (181h45m46.685434713s) [noise=true , address=145.79.227.20:4439 , piece_id=PLNGHRV2N3GPQYYSGU7TKL2FMFCU4JQWXPM4K4HUHH3WKKLLQYVQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.146016122 (181h45m46.697486573s) [noise=true , address=145.79.227.17:4971 , piece_id=LBNF3Z5A3NZGUQKCWAIW4XPONBC2DD324VXP44FFCQ5FEXA5FZOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.153123927 (181h45m46.704594388s) [noise=true , address=145.79.227.31:4798 , piece_id=MBH7EMPFU4UKRIDVAHD6B7COGTRYBUZ3XS2L2UVCJ5ONXEJE4POQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.174476267 (181h45m46.725946728s) [noise=true , address=145.79.227.9:4366 , piece_id=SI7YJXWUTUXDVPNBYCLDPPVRE66RXHLPBLB43A45TND34UO4TF7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.197067667 (181h45m46.748538128s) [noise=true , address=145.79.227.14:4837 , piece_id=YBBFYSC22KSQK5GENOT2NU624PDLUNX2TWQ24RNRW3WMGJT6P7RA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238467652 (181h45m46.789938113s) [noise=true , address=77.90.31.14:4663 , piece_id=W4MIXY47CPR4IYA6WCREO4VC5BVGAFBJB2BJV5ZNODIIHEXJT2VQ, seg_pos={0 0}, upload=160073]: Done uploading piece. err: <nil> Nov 16 21:49:48.238492430 (181h45m46.789962891s) [seg_pos={0 0}, upload=160073]: Segment reached optimal threshold of 13 pieces. Nov 16 21:49:48.238743558 (181h45m46.790214019s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238753988 (181h45m46.790224449s) [noise=true , address=45.91.116.14:4190 , piece_id=NWVY4TNZTT2Z3OBXPPYXOAQQALIHMGWML2QQLIREWBDHT3GBUDOA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.238823801 (181h45m46.790294262s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.238844991 (181h45m46.790315442s) [noise=true , address=45.91.116.4:4939 , piece_id=OFWZ2DS67MBSCO7MJ3NYCCUOEBFNDDZ64GSB3XNZ2J4UZJW6BX7Q, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239032308 (181h45m46.790502769s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Nov 16 21:49:48.239041596 (181h45m46.790512047s) [noise=true , address=45.91.116.6:4975 , piece_id=6PPD4GO4CNNX3A246GZQRXMBUFCKUPHI2L5JAOSNYWXUFBRM4SEA, seg_pos={0 0}, upload=160073]: Long tail cancellation due to optimal threshold Nov 16 21:49:48.239195508 (181h45m46.790665959s) [seg_pos={0 0}, upload=160073]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 16 21:49:48.239303494 (181h45m46.790773955s) [upload=160073]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 16 21:49:48.239313143 (181h45m46.790783594s) [upload=160073]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 16 21:49:53.369411932 (181h45m51.920882393s) [upload=160073]: Done uploading segments... Nov 16 21:49:53.369436949 (181h45m51.92090741s) [upload=160073]: Upload finished. err: <nil> Duration: 5.931330987s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.094043035s trace: https://jaeger.dev.storj.io/trace/1533f838bd5c565 FAIL (2025-11-16 22:07:53 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.094070396s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.416856985s trace: https://jaeger.dev.storj.io/trace/ef0652f4a47c08 FAIL (2025-11-16 22:07:48 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.416886582s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)