Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.......X........X.....X.XX...........................X....X.] duration: 36.144952327s trace: https://jaeger.dev.storj.io/trace/3991b0e55a564d7a FAIL (2025-12-17 14:45:15 UTC) orbiter warning: context deadline exceeded Duration: 36.144975261s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 16.428284418s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:45:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 16.428328952s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.863714855s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:44:48 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.863744281s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.015474553s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:44:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 6.01550355s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [........X........X.....X.XX...........................X....X] duration: 1m0.001113128s trace: https://jaeger.dev.storj.io/trace/3991b0e55a564d7a FAIL (2025-12-17 14:44:39 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001135342s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 34 out of 60 is failed (threshold=3) [XXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X..X.XX] duration: 30.001175017s trace: https://jaeger.dev.storj.io/trace/68c84c95ab8d8d79 FAIL (2025-12-17 14:44:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001196459s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 50 out of 60 is failed (threshold=6) [XXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX..XXXX.] duration: 3.975805079s trace: https://jaeger.dev.storj.io/trace/c1a8fb2c7270278 FAIL (2025-12-17 14:44:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.975842982s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 8.75335084s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:44:10 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.753377762s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 34 out of 60 is failed (threshold=3) [XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X..X.X] duration: 30.001555849s trace: https://jaeger.dev.storj.io/trace/66bad4cb7bd56841 FAIL (2025-12-17 14:43:57 UTC) orbiter warning: context deadline exceeded Duration: 30.001566629s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.994438298s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:43:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 5.994455542s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 51 out of 60 is failed (threshold=6) [XXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX..XXXX] duration: 5.001025426s trace: https://jaeger.dev.storj.io/trace/c1a8fb2c7270278 FAIL (2025-12-17 14:43:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001040637s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X..X.] duration: 28.416182836s trace: https://jaeger.dev.storj.io/trace/72748f2d06a953a6 FAIL (2025-12-17 14:43:26 UTC) orbiter warning: context deadline exceeded Duration: 28.416211059s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 23.931394211s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:43:22 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 23.931423947s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.823325999s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:43:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 5.823351267s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.........X........X.....X.XX...........................X....] duration: 21.56733963s trace: https://jaeger.dev.storj.io/trace/5a65f99cf6ecdfc4 FAIL (2025-12-17 14:42:56 UTC) orbiter warning: context deadline exceeded Duration: 21.56736686s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 34 out of 60 is failed (threshold=3) [X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X..X] duration: 30.001005679s trace: https://jaeger.dev.storj.io/trace/72748f2d06a953a6 FAIL (2025-12-17 14:42:49 UTC) orbiter warning: context deadline exceeded Duration: 30.001029165s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 51 out of 60 is failed (threshold=6) [XXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX..XXX] duration: 5.001566368s trace: https://jaeger.dev.storj.io/trace/31a8514144116537 FAIL (2025-12-17 14:42:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00159364s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.883714931s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:42:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 5.883741742s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 51 out of 60 is failed (threshold=6) [XXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX..XX] duration: 5.002574307s trace: https://jaeger.dev.storj.io/trace/4b67095560c2ecc4 FAIL (2025-12-17 14:42:07 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002599704s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.933262083s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:42:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.933292431s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..........X........X.....X.XX...........................X...] duration: 9.788573435s trace: https://jaeger.dev.storj.io/trace/5a65f99cf6ecdfc4 FAIL (2025-12-17 14:42:02 UTC) orbiter warning: context deadline exceeded Duration: 9.788590407s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X..] duration: 19.965239711s trace: https://jaeger.dev.storj.io/trace/1770fdaf0ae5cdd4 FAIL (2025-12-17 14:42:00 UTC) orbiter warning: context deadline exceeded Duration: 19.96527095s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...........X........X.....X.XX...........................X..] duration: 31.549156395s trace: https://jaeger.dev.storj.io/trace/5a65f99cf6ecdfc4 FAIL (2025-12-17 14:41:52 UTC) orbiter warning: context deadline exceeded Duration: 31.549173998s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.506703694s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:41:38 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.506740804s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 34 out of 60 is failed (threshold=3) [X.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X.] duration: 25.633435892s trace: https://jaeger.dev.storj.io/trace/1770fdaf0ae5cdd4 FAIL (2025-12-17 14:41:34 UTC) orbiter warning: context deadline exceeded Duration: 25.633453406s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 51 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX..X] duration: 5.002095976s trace: https://jaeger.dev.storj.io/trace/158b956ce87462a9 FAIL (2025-12-17 14:41:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002127566s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.779958114s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:41:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 6.779987311s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [............X........X.....X.XX...........................X.] duration: 40.76326226s trace: https://jaeger.dev.storj.io/trace/5a65f99cf6ecdfc4 FAIL (2025-12-17 14:41:20 UTC) orbiter warning: context deadline exceeded Duration: 40.763276287s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.186765408s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:41:13 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.186800394s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 34 out of 60 is failed (threshold=3) [.X.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.X] duration: 30.000859401s trace: https://jaeger.dev.storj.io/trace/1770fdaf0ae5cdd4 FAIL (2025-12-17 14:41:08 UTC) orbiter warning: context deadline exceeded Duration: 30.000882044s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 51 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX..] duration: 4.076163279s trace: https://jaeger.dev.storj.io/trace/68abbe65f2450eb4 FAIL (2025-12-17 14:41:08 UTC) orbiter warning: context deadline exceeded Duration: 4.07619046s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 10.900913223s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:40:51 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 10.900956484s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 52 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX.] duration: 3.866828706s trace: https://jaeger.dev.storj.io/trace/68abbe65f2450eb4 FAIL (2025-12-17 14:40:49 UTC) orbiter warning: context deadline exceeded Duration: 3.866856329s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.............X........X.....X.XX...........................X] duration: 1m0.002010257s trace: https://jaeger.dev.storj.io/trace/5a65f99cf6ecdfc4 FAIL (2025-12-17 14:40:39 UTC) orbiter warning: context deadline exceeded Duration: 1m0.002023672s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.021618343s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:40:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 6.021654643s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [..X.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X.] duration: 27.737687963s trace: https://jaeger.dev.storj.io/trace/58a97aaecb0df206 FAIL (2025-12-17 14:40:22 UTC) orbiter warning: context deadline exceeded Duration: 27.737717529s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 18.950662665s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:40:01 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 18.950697661s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXXX] duration: 5.001216452s trace: https://jaeger.dev.storj.io/trace/68abbe65f2450eb4 FAIL (2025-12-17 14:39:39 UTC) orbiter warning: context deadline exceeded Duration: 5.001240717s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..............X........X.....X.XX...........................] duration: 49.523483745s trace: https://jaeger.dev.storj.io/trace/4beef8fb89704bac FAIL (2025-12-17 14:39:39 UTC) orbiter warning: context deadline exceeded Duration: 49.523510366s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 8.317283637s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:39:22 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.317315088s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXXX] duration: 5.001095964s trace: https://jaeger.dev.storj.io/trace/21484b5e0571554c FAIL (2025-12-17 14:39:21 UTC) orbiter warning: context deadline exceeded Duration: 5.001123125s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 11.634539795s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:39:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 11.634571495s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [...X.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.X] duration: 30.002123635s trace: https://jaeger.dev.storj.io/trace/58a97aaecb0df206 FAIL (2025-12-17 14:39:14 UTC) orbiter warning: context deadline exceeded Duration: 30.002150405s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 14.58365921s trace: https://jaeger.dev.storj.io/trace/269f3a09cefa5304 FAIL (2025-12-17 14:38:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA) context deadline exceeded; piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA) context deadline exceeded ======LOGS====== Dec 17 14:23:59.875948279 (286h54m59.747031314s) [upload=251555]: Starting upload Dec 17 14:23:59.876005657 (286h54m59.747088692s) [upload=251555]: Done starting upload Dec 17 14:23:59.876197153 (286h54m59.747280198s) [upload=251555]: Uploading segments... Dec 17 14:23:59.876299007 (286h54m59.747382052s) [upload=251555]: Got next segment. Inline: false Dec 17 14:23:59.876339674 (286h54m59.747422719s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 17 14:23:59.876346406 (286h54m59.747429452s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 17 14:24:00.124613924 (286h54m59.995696989s) [seg_pos={0 0}, upload=251555]: Begin upload segment... Dec 17 14:24:00.124676081 (286h54m59.995759126s) [seg_pos={0 0}, upload=251555]: Done begin upload segment. Dec 17 14:24:00.124715056 (286h54m59.995798102s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.124826919 (286h54m59.995909964s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125375444 (286h54m59.996458489s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125451969 (286h54m59.996535005s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125707846 (286h54m59.996790891s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.125863774 (286h54m59.996946819s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126111014 (286h54m59.99719406s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126201967 (286h54m59.997285012s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126456553 (286h54m59.997539588s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126581640 (286h54m59.997664685s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.126756253 (286h54m59.997839288s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127169409 (286h54m59.998252455s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127195309 (286h54m59.998278354s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127489889 (286h54m59.998572925s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.127604368 (286h54m59.998687403s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:00.128446622 (286h54m59.999529657s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Uploading piece... Dec 17 14:24:01.745844898 (286h55m1.616927943s) [upload=251555]: Next returned nil segment Dec 17 14:24:01.745865247 (286h55m1.616948292s) [upload=251555]: Waiting for error group managing segments... Dec 17 14:24:02.125399239 (286h55m1.996482284s) [noise=true , address=145.79.227.36:4518 , piece_id=GXBP4F3BZ2ZAFU7L6F2ORN6V432VBRVNBYQN3GLWDH727HSB7NIA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.126481550 (286h55m1.997564605s) [noise=true , address=145.79.227.25:4617 , piece_id=76BUQX3JFTWRCFO77QSN36X7GI5UFDDBCOWO2PUNGMTB7XW7XJEA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.127574442 (286h55m1.998657487s) [noise=true , address=145.79.227.8:4330 , piece_id=7VA57PV37XE2PLTLZSTFXAL3DWIGUPCLUDGLLNZFWF2L2C7QIOUQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144487878 (286h55m2.015570923s) [noise=true , address=145.79.227.48:4301 , piece_id=NWFW5XTFWAGTRK2XHCRHP4VS7RLQ2KMWFOPPL3OR2ZLBU4YU736A, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.144919459 (286h55m2.016002504s) [noise=true , address=145.79.227.31:4799 , piece_id=UEN4CLHW576UIF425PEM4K5ZX3E3HQYLOY55M72ILEIZBTI4GBBQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.206563161 (286h55m2.077646206s) [noise=true , address=145.79.227.37:4565 , piece_id=T6TKEWR2AJ4DTQ4EUMZMYQXQ7TTHBCREXJDDUDWTZYP7YHL23EKA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.220071653 (286h55m2.091154698s) [noise=true , address=145.79.227.43:4157 , piece_id=QL4JPEWPCU3OQZGYTUTIOYZ2PMIQCTA4CFEP7VNAQL77DUJYKJWQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.223129416 (286h55m2.094212461s) [noise=true , address=145.79.227.19:4077 , piece_id=D6BURLEYHSFWR3SFC4WQEA23MW22HKKXIBCWUM42HAWF4IAL6TRQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232568790 (286h55m2.103651865s) [noise=true , address=145.79.227.6:4258 , piece_id=ZUPRHDSYFW5Y7QW2L3MBMHYN4HSHIM6EUAK4LS3VPJGSGFMFR2AQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.232747801 (286h55m2.103830846s) [noise=true , address=145.79.227.41:4087 , piece_id=TYT6PNIZCSFFBEPH4ZO56BZ655ZKXBB57EHORC6Q3XVMDHA5TGJQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.233412857 (286h55m2.104495902s) [noise=true , address=145.79.227.39:4015 , piece_id=INDYRD3AU6DM5MB2AZLBYR5UCGUDMJLSLSAWDOO24GVFDFVGVJKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.402580213 (286h55m2.273663248s) [noise=true , address=45.91.116.8:4048 , piece_id=4V3JN6SRF23AHT7P4ZWBK2WIGDC7NFCT37UTSII3M2IJEXJXJMRA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407473961 (286h55m2.278557026s) [noise=true , address=45.91.116.7:4011 , piece_id=JOFAO3BWRKAMKOKIJLUHASUHUWF72AWCBNO3UQZ6KMPAGC7QQK6Q, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.407622134 (286h55m2.278705179s) [seg_pos={0 0}, upload=251555]: Segment reached optimal threshold of 13 pieces. Dec 17 14:24:02.407935050 (286h55m2.279018095s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.407948014 (286h55m2.279031059s) [noise=true , address=77.90.31.16:4736 , piece_id=76FWJER4KP2C7AQCPIHSXR5GTKKPWIJFIVUWBGYX3YOBCLF6TEKQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408286699 (286h55m2.279369744s) [noise=true , address=45.91.116.9:4263 , piece_id=YF6RQ5CVDKT6K4FHI7N34TBB2O5NW2CEIEQMAJ7WHV5EHH542ETA, seg_pos={0 0}, upload=251555]: Done uploading piece. err: <nil> Dec 17 14:24:02.408326715 (286h55m2.279409761s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: expected piece hash; context canceled; EOF Dec 17 14:24:02.408338097 (286h55m2.279421142s) [noise=true , address=45.91.116.11:4120 , piece_id=XGJXD6MDPKR6HZJCGCI7TX5CLPITCGLQEDQRKMC3GDST53AFWTOQ, seg_pos={0 0}, upload=251555]: Long tail cancellation due to optimal threshold Dec 17 14:24:02.408367493 (286h55m2.279450538s) [seg_pos={0 0}, upload=251555]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 17 14:24:02.408459939 (286h55m2.279542984s) [upload=251555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 17 14:24:02.408466732 (286h55m2.279549777s) [upload=251555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 17 14:24:02.694441231 (286h55m2.565524286s) [upload=251555]: Done uploading segments... Dec 17 14:24:02.694468864 (286h55m2.565551899s) [upload=251555]: Upload finished. err: <nil> Duration: 14.583684538s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [X...X.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX.] duration: 24.218505176s trace: https://jaeger.dev.storj.io/trace/24188551663f148d FAIL (2025-12-17 14:38:24 UTC) orbiter warning: context deadline exceeded Duration: 24.218548339s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 52 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXXX] duration: 5.001145674s trace: https://jaeger.dev.storj.io/trace/326f8cc1d0648a1d FAIL (2025-12-17 14:38:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001285241s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 8.932445644s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:38:18 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.932471353s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 33 out of 60 is failed (threshold=3) [.X...X.X.XXXXXXXXXXX.X...XX.X...X....XXX.X.X.X.XXX.X..X.X.XX] duration: 30.000819777s trace: https://jaeger.dev.storj.io/trace/24188551663f148d FAIL (2025-12-17 14:38:00 UTC) orbiter warning: context deadline exceeded Duration: 30.000846578s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 52 out of 60 is failed (threshold=6) [X.XXXXXXXXXXXXXXXXX.XXXXX.XXXXXXXXXXXXX.X.XXXX.XX.XXXXX.XXXX] duration: 5.001037491s trace: https://jaeger.dev.storj.io/trace/7fcfb814cd2831a4 FAIL (2025-12-17 14:37:59 UTC) orbiter warning: context deadline exceeded Duration: 5.001060073s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 8.579718051s trace: https://jaeger.dev.storj.io/trace/3a4b0fa83b6b5dd9 FAIL (2025-12-17 14:37:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.579837017s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)