Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.809957847s trace: https://jaeger.dev.storj.io/trace/6a9c24d5f746bcd9 FAIL (2026-01-10 21:20:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 10 21:16:13.826292475 (869h47m13.69737552s) [upload=760991]: Starting upload Jan 10 21:16:13.826511804 (869h47m13.697594889s) [upload=760991]: Done starting upload Jan 10 21:16:13.826578310 (869h47m13.697661345s) [upload=760991]: Uploading segments... Jan 10 21:16:13.826879934 (869h47m13.697962989s) [upload=760991]: Got next segment. Inline: false Jan 10 21:16:13.826947262 (869h47m13.698030348s) [upload=760991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 10 21:16:13.826953965 (869h47m13.69803702s) [upload=760991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 10 21:16:14.071989063 (869h47m13.943072098s) [seg_pos={0 0}, upload=760991]: Begin upload segment... Jan 10 21:16:14.072045660 (869h47m13.943128695s) [seg_pos={0 0}, upload=760991]: Done begin upload segment. Jan 10 21:16:14.072106606 (869h47m13.943189652s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072168765 (869h47m13.94325181s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072225392 (869h47m13.943308427s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072854670 (869h47m13.943937705s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.073407874 (869h47m13.944490909s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074027234 (869h47m13.945110269s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074506166 (869h47m13.945589201s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.076055455 (869h47m13.9471385s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077873718 (869h47m13.948956753s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077930205 (869h47m13.94901325s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.079218408 (869h47m13.950301443s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.080991795 (869h47m13.95207484s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.082276882 (869h47m13.953359917s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.086152672 (869h47m13.957235717s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.088176636 (869h47m13.959259681s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.091296346 (869h47m13.962379381s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:15.777276272 (869h47m15.648359317s) [upload=760991]: Next returned nil segment Jan 10 21:16:15.777289137 (869h47m15.648372172s) [upload=760991]: Waiting for error group managing segments... Jan 10 21:16:16.049459864 (869h47m15.920542909s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.062528699 (869h47m15.933611734s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.094641115 (869h47m15.96572415s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.095532593 (869h47m15.966615648s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.104522481 (869h47m15.975605516s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.144663855 (869h47m16.01574689s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.185374913 (869h47m16.056457948s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.213186510 (869h47m16.084269555s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.280050617 (869h47m16.151133662s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.282952472 (869h47m16.154035517s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.296480753 (869h47m16.167563798s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.380183644 (869h47m16.251266689s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:43.135289001 (869h47m43.006372046s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection Jan 10 21:16:43.135366619 (869h47m43.006449654s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection Jan 10 21:16:43.135428196 (869h47m43.006511231s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12S5FyCphEYSuT8Bz2aZhiRkm6PYDnhD8NmdMQuXGscbMfaaeis, address:145.79.227.45:4230): protocol: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection Jan 10 21:16:43.135438275 (869h47m43.00652132s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection Jan 10 21:16:43.135459575 (869h47m43.00654261s) [seg_pos={0 0}, upload=760991]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 10 21:16:43.374310077 (869h47m43.245393113s) [upload=760991]: Done uploading segments... Jan 10 21:16:43.374344443 (869h47m43.245427478s) [upload=760991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.809984017s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [X....X.X....XX.........X....X...............................] duration: 3.49466044s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:19:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.494684345s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 14.345906505s trace: https://jaeger.dev.storj.io/trace/6a9c24d5f746bcd9 FAIL (2026-01-10 21:19:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 10 21:16:13.826292475 (869h47m13.69737552s) [upload=760991]: Starting upload Jan 10 21:16:13.826511804 (869h47m13.697594889s) [upload=760991]: Done starting upload Jan 10 21:16:13.826578310 (869h47m13.697661345s) [upload=760991]: Uploading segments... Jan 10 21:16:13.826879934 (869h47m13.697962989s) [upload=760991]: Got next segment. Inline: false Jan 10 21:16:13.826947262 (869h47m13.698030348s) [upload=760991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 10 21:16:13.826953965 (869h47m13.69803702s) [upload=760991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 10 21:16:14.071989063 (869h47m13.943072098s) [seg_pos={0 0}, upload=760991]: Begin upload segment... Jan 10 21:16:14.072045660 (869h47m13.943128695s) [seg_pos={0 0}, upload=760991]: Done begin upload segment. Jan 10 21:16:14.072106606 (869h47m13.943189652s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072168765 (869h47m13.94325181s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072225392 (869h47m13.943308427s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072854670 (869h47m13.943937705s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.073407874 (869h47m13.944490909s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074027234 (869h47m13.945110269s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074506166 (869h47m13.945589201s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.076055455 (869h47m13.9471385s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077873718 (869h47m13.948956753s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077930205 (869h47m13.94901325s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.079218408 (869h47m13.950301443s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.080991795 (869h47m13.95207484s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.082276882 (869h47m13.953359917s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.086152672 (869h47m13.957235717s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.088176636 (869h47m13.959259681s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.091296346 (869h47m13.962379381s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:15.777276272 (869h47m15.648359317s) [upload=760991]: Next returned nil segment Jan 10 21:16:15.777289137 (869h47m15.648372172s) [upload=760991]: Waiting for error group managing segments... Jan 10 21:16:16.049459864 (869h47m15.920542909s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.062528699 (869h47m15.933611734s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.094641115 (869h47m15.96572415s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.095532593 (869h47m15.966615648s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.104522481 (869h47m15.975605516s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.144663855 (869h47m16.01574689s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.185374913 (869h47m16.056457948s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.213186510 (869h47m16.084269555s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.280050617 (869h47m16.151133662s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.282952472 (869h47m16.154035517s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.296480753 (869h47m16.167563798s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.380183644 (869h47m16.251266689s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:43.135289001 (869h47m43.006372046s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection Jan 10 21:16:43.135366619 (869h47m43.006449654s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection Jan 10 21:16:43.135428196 (869h47m43.006511231s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12S5FyCphEYSuT8Bz2aZhiRkm6PYDnhD8NmdMQuXGscbMfaaeis, address:145.79.227.45:4230): protocol: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection Jan 10 21:16:43.135438275 (869h47m43.00652132s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection Jan 10 21:16:43.135459575 (869h47m43.00654261s) [seg_pos={0 0}, upload=760991]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 10 21:16:43.374310077 (869h47m43.245393113s) [upload=760991]: Done uploading segments... Jan 10 21:16:43.374344443 (869h47m43.245427478s) [upload=760991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 14.345934369s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.702404332s trace: https://jaeger.dev.storj.io/trace/6a9c24d5f746bcd9 FAIL (2026-01-10 21:18:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 10 21:16:13.826292475 (869h47m13.69737552s) [upload=760991]: Starting upload Jan 10 21:16:13.826511804 (869h47m13.697594889s) [upload=760991]: Done starting upload Jan 10 21:16:13.826578310 (869h47m13.697661345s) [upload=760991]: Uploading segments... Jan 10 21:16:13.826879934 (869h47m13.697962989s) [upload=760991]: Got next segment. Inline: false Jan 10 21:16:13.826947262 (869h47m13.698030348s) [upload=760991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 10 21:16:13.826953965 (869h47m13.69803702s) [upload=760991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 10 21:16:14.071989063 (869h47m13.943072098s) [seg_pos={0 0}, upload=760991]: Begin upload segment... Jan 10 21:16:14.072045660 (869h47m13.943128695s) [seg_pos={0 0}, upload=760991]: Done begin upload segment. Jan 10 21:16:14.072106606 (869h47m13.943189652s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072168765 (869h47m13.94325181s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072225392 (869h47m13.943308427s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072854670 (869h47m13.943937705s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.073407874 (869h47m13.944490909s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074027234 (869h47m13.945110269s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074506166 (869h47m13.945589201s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.076055455 (869h47m13.9471385s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077873718 (869h47m13.948956753s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077930205 (869h47m13.94901325s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.079218408 (869h47m13.950301443s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.080991795 (869h47m13.95207484s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.082276882 (869h47m13.953359917s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.086152672 (869h47m13.957235717s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.088176636 (869h47m13.959259681s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.091296346 (869h47m13.962379381s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:15.777276272 (869h47m15.648359317s) [upload=760991]: Next returned nil segment Jan 10 21:16:15.777289137 (869h47m15.648372172s) [upload=760991]: Waiting for error group managing segments... Jan 10 21:16:16.049459864 (869h47m15.920542909s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.062528699 (869h47m15.933611734s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.094641115 (869h47m15.96572415s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.095532593 (869h47m15.966615648s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.104522481 (869h47m15.975605516s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.144663855 (869h47m16.01574689s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.185374913 (869h47m16.056457948s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.213186510 (869h47m16.084269555s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.280050617 (869h47m16.151133662s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.282952472 (869h47m16.154035517s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.296480753 (869h47m16.167563798s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.380183644 (869h47m16.251266689s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:43.135289001 (869h47m43.006372046s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection Jan 10 21:16:43.135366619 (869h47m43.006449654s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection Jan 10 21:16:43.135428196 (869h47m43.006511231s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12S5FyCphEYSuT8Bz2aZhiRkm6PYDnhD8NmdMQuXGscbMfaaeis, address:145.79.227.45:4230): protocol: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection Jan 10 21:16:43.135438275 (869h47m43.00652132s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection Jan 10 21:16:43.135459575 (869h47m43.00654261s) [seg_pos={0 0}, upload=760991]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 10 21:16:43.374310077 (869h47m43.245393113s) [upload=760991]: Done uploading segments... Jan 10 21:16:43.374344443 (869h47m43.245427478s) [upload=760991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.702432436s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.X....X.X....XX.........X....X..............................] duration: 3.469632396s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:18:48 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.469667073s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 7.412525552s trace: https://jaeger.dev.storj.io/trace/6a9c24d5f746bcd9 FAIL (2026-01-10 21:18:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 10 21:16:13.826292475 (869h47m13.69737552s) [upload=760991]: Starting upload Jan 10 21:16:13.826511804 (869h47m13.697594889s) [upload=760991]: Done starting upload Jan 10 21:16:13.826578310 (869h47m13.697661345s) [upload=760991]: Uploading segments... Jan 10 21:16:13.826879934 (869h47m13.697962989s) [upload=760991]: Got next segment. Inline: false Jan 10 21:16:13.826947262 (869h47m13.698030348s) [upload=760991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 10 21:16:13.826953965 (869h47m13.69803702s) [upload=760991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 10 21:16:14.071989063 (869h47m13.943072098s) [seg_pos={0 0}, upload=760991]: Begin upload segment... Jan 10 21:16:14.072045660 (869h47m13.943128695s) [seg_pos={0 0}, upload=760991]: Done begin upload segment. Jan 10 21:16:14.072106606 (869h47m13.943189652s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072168765 (869h47m13.94325181s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072225392 (869h47m13.943308427s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072854670 (869h47m13.943937705s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.073407874 (869h47m13.944490909s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074027234 (869h47m13.945110269s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074506166 (869h47m13.945589201s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.076055455 (869h47m13.9471385s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077873718 (869h47m13.948956753s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077930205 (869h47m13.94901325s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.079218408 (869h47m13.950301443s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.080991795 (869h47m13.95207484s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.082276882 (869h47m13.953359917s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.086152672 (869h47m13.957235717s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.088176636 (869h47m13.959259681s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.091296346 (869h47m13.962379381s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:15.777276272 (869h47m15.648359317s) [upload=760991]: Next returned nil segment Jan 10 21:16:15.777289137 (869h47m15.648372172s) [upload=760991]: Waiting for error group managing segments... Jan 10 21:16:16.049459864 (869h47m15.920542909s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.062528699 (869h47m15.933611734s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.094641115 (869h47m15.96572415s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.095532593 (869h47m15.966615648s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.104522481 (869h47m15.975605516s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.144663855 (869h47m16.01574689s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.185374913 (869h47m16.056457948s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.213186510 (869h47m16.084269555s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.280050617 (869h47m16.151133662s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.282952472 (869h47m16.154035517s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.296480753 (869h47m16.167563798s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.380183644 (869h47m16.251266689s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:43.135289001 (869h47m43.006372046s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection Jan 10 21:16:43.135366619 (869h47m43.006449654s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection Jan 10 21:16:43.135428196 (869h47m43.006511231s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12S5FyCphEYSuT8Bz2aZhiRkm6PYDnhD8NmdMQuXGscbMfaaeis, address:145.79.227.45:4230): protocol: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection Jan 10 21:16:43.135438275 (869h47m43.00652132s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection Jan 10 21:16:43.135459575 (869h47m43.00654261s) [seg_pos={0 0}, upload=760991]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 10 21:16:43.374310077 (869h47m43.245393113s) [upload=760991]: Done uploading segments... Jan 10 21:16:43.374344443 (869h47m43.245427478s) [upload=760991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.412547303s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [X.X....X.X....XX.........X....X.............................] duration: 3.556813461s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:17:46 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.5568439s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [XX.X....X.X....XX.........X....X............................] duration: 3.578455726s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:17:07 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.578482808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.929448302s trace: https://jaeger.dev.storj.io/trace/6a9c24d5f746bcd9 FAIL (2026-01-10 21:17:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 10 21:16:13.826292475 (869h47m13.69737552s) [upload=760991]: Starting upload Jan 10 21:16:13.826511804 (869h47m13.697594889s) [upload=760991]: Done starting upload Jan 10 21:16:13.826578310 (869h47m13.697661345s) [upload=760991]: Uploading segments... Jan 10 21:16:13.826879934 (869h47m13.697962989s) [upload=760991]: Got next segment. Inline: false Jan 10 21:16:13.826947262 (869h47m13.698030348s) [upload=760991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 10 21:16:13.826953965 (869h47m13.69803702s) [upload=760991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 10 21:16:14.071989063 (869h47m13.943072098s) [seg_pos={0 0}, upload=760991]: Begin upload segment... Jan 10 21:16:14.072045660 (869h47m13.943128695s) [seg_pos={0 0}, upload=760991]: Done begin upload segment. Jan 10 21:16:14.072106606 (869h47m13.943189652s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072168765 (869h47m13.94325181s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072225392 (869h47m13.943308427s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072854670 (869h47m13.943937705s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.073407874 (869h47m13.944490909s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074027234 (869h47m13.945110269s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074506166 (869h47m13.945589201s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.076055455 (869h47m13.9471385s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077873718 (869h47m13.948956753s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077930205 (869h47m13.94901325s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.079218408 (869h47m13.950301443s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.080991795 (869h47m13.95207484s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.082276882 (869h47m13.953359917s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.086152672 (869h47m13.957235717s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.088176636 (869h47m13.959259681s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.091296346 (869h47m13.962379381s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:15.777276272 (869h47m15.648359317s) [upload=760991]: Next returned nil segment Jan 10 21:16:15.777289137 (869h47m15.648372172s) [upload=760991]: Waiting for error group managing segments... Jan 10 21:16:16.049459864 (869h47m15.920542909s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.062528699 (869h47m15.933611734s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.094641115 (869h47m15.96572415s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.095532593 (869h47m15.966615648s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.104522481 (869h47m15.975605516s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.144663855 (869h47m16.01574689s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.185374913 (869h47m16.056457948s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.213186510 (869h47m16.084269555s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.280050617 (869h47m16.151133662s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.282952472 (869h47m16.154035517s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.296480753 (869h47m16.167563798s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.380183644 (869h47m16.251266689s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:43.135289001 (869h47m43.006372046s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection Jan 10 21:16:43.135366619 (869h47m43.006449654s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection Jan 10 21:16:43.135428196 (869h47m43.006511231s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12S5FyCphEYSuT8Bz2aZhiRkm6PYDnhD8NmdMQuXGscbMfaaeis, address:145.79.227.45:4230): protocol: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection Jan 10 21:16:43.135438275 (869h47m43.00652132s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection Jan 10 21:16:43.135459575 (869h47m43.00654261s) [seg_pos={0 0}, upload=760991]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 10 21:16:43.374310077 (869h47m43.245393113s) [upload=760991]: Done uploading segments... Jan 10 21:16:43.374344443 (869h47m43.245427478s) [upload=760991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.929476115s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [.XX.X....X.X....XX.........X....X...........................] duration: 3.654250832s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:16:47 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.654277393s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.240454539s trace: https://jaeger.dev.storj.io/trace/6a9c24d5f746bcd9 FAIL (2026-01-10 21:16:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 10 21:16:13.826292475 (869h47m13.69737552s) [upload=760991]: Starting upload Jan 10 21:16:13.826511804 (869h47m13.697594889s) [upload=760991]: Done starting upload Jan 10 21:16:13.826578310 (869h47m13.697661345s) [upload=760991]: Uploading segments... Jan 10 21:16:13.826879934 (869h47m13.697962989s) [upload=760991]: Got next segment. Inline: false Jan 10 21:16:13.826947262 (869h47m13.698030348s) [upload=760991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 10 21:16:13.826953965 (869h47m13.69803702s) [upload=760991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 10 21:16:14.071989063 (869h47m13.943072098s) [seg_pos={0 0}, upload=760991]: Begin upload segment... Jan 10 21:16:14.072045660 (869h47m13.943128695s) [seg_pos={0 0}, upload=760991]: Done begin upload segment. Jan 10 21:16:14.072106606 (869h47m13.943189652s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072168765 (869h47m13.94325181s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072225392 (869h47m13.943308427s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.072854670 (869h47m13.943937705s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.073407874 (869h47m13.944490909s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074027234 (869h47m13.945110269s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.074506166 (869h47m13.945589201s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.076055455 (869h47m13.9471385s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077873718 (869h47m13.948956753s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.077930205 (869h47m13.94901325s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.079218408 (869h47m13.950301443s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.080991795 (869h47m13.95207484s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.082276882 (869h47m13.953359917s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.086152672 (869h47m13.957235717s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.088176636 (869h47m13.959259681s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:14.091296346 (869h47m13.962379381s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Uploading piece... Jan 10 21:16:15.777276272 (869h47m15.648359317s) [upload=760991]: Next returned nil segment Jan 10 21:16:15.777289137 (869h47m15.648372172s) [upload=760991]: Waiting for error group managing segments... Jan 10 21:16:16.049459864 (869h47m15.920542909s) [noise=true , address=77.90.31.19:4841 , piece_id=BVC364OVF2LFKGHC55FLWO7GXD263QREIYS4673AMPIZ57V4APDQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.062528699 (869h47m15.933611734s) [noise=true , address=145.79.227.3:4151 , piece_id=DFJ4WYHRSFWCSPVCWKAWTFIKY3IXGCY5WBEJ7B32ICJ7I3K7CYSQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.094641115 (869h47m15.96572415s) [noise=true , address=145.79.227.49:4339 , piece_id=LRDM6XTPQ4Z54O3VHSIGW2CTMCNO3XQQNTWOPERZNCGGZZAMASRA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.095532593 (869h47m15.966615648s) [noise=true , address=145.79.227.42:4122 , piece_id=PJ73L6UVGYXPJISPZDVY2VRLKR4KDGNR223QWPJFX2BKXH7JQLQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.104522481 (869h47m15.975605516s) [noise=true , address=145.79.227.44:4195 , piece_id=FDQ7C64JE5AW2SVV3GIEQKC35WOMPTGJCNBMVSGSGARJXRMQBCQQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.144663855 (869h47m16.01574689s) [noise=true , address=145.79.227.23:4547 , piece_id=2CAY5TD7J4YK4HP3TJRKN6ILOGVVUCCEHFGJG5NOZHYRWJRHFXCA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.185374913 (869h47m16.056457948s) [noise=true , address=45.91.116.14:4190 , piece_id=QL7ARL4DRLUVA2AR65JAFFXHMRCKPMMNVTCS2OREA4OQA6ORB4QA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.213186510 (869h47m16.084269555s) [noise=true , address=45.91.116.9:4264 , piece_id=TIXGRIQZYNITJC3TC5UUIW77OAGMPSOBPUGMTT27EMYFUQUR7GUA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.280050617 (869h47m16.151133662s) [noise=true , address=145.79.227.12:4765 , piece_id=4HYFSGAPAERJRQM7ON3HAGOTZQ46DVAVA44DWP2X3D6B4XZZMZEA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.282952472 (869h47m16.154035517s) [noise=true , address=145.79.227.4:4185 , piece_id=JOFA7SKZPGO4FMJPVRIDIX2OSDII6Q5JCCZHXRSTKPG2VRARDVXQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.296480753 (869h47m16.167563798s) [noise=true , address=145.79.227.14:4837 , piece_id=PXYONX5YWMMIOKG3KBU5XVLIW3AKWCGC732IQ6IRSF3CD5KVGTLA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:16.380183644 (869h47m16.251266689s) [noise=true , address=145.79.227.38:4979 , piece_id=7BTHTDLV2I42E7NWV27S5IYHDYNLLMFFAVRLW6NWBVFPTJNSALOQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: <nil> Jan 10 21:16:43.135289001 (869h47m43.006372046s) [noise=true , address=145.79.227.52:4447 , piece_id=Y5TPVKXZM2UXSUR2UHLEY6GAYRVDFD3PPKQYRSEFLWIQAO5RG4IQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55494->145.79.227.52:4447: use of closed network connection Jan 10 21:16:43.135366619 (869h47m43.006449654s) [noise=true , address=145.79.227.34:4905 , piece_id=VBIZHM3HVDZVICJ7BJNXLJPOYQ276LZ6XEXIXFXZGWDBA4TAPFFA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59106->145.79.227.34:4905: use of closed network connection Jan 10 21:16:43.135428196 (869h47m43.006511231s) [noise=true , address=145.79.227.45:4230 , piece_id=EVPT4ATUOELX7ZOVNPSJFCYV6YJTB5NXRYGUOL5X4UCGLHPNYNYA, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:12S5FyCphEYSuT8Bz2aZhiRkm6PYDnhD8NmdMQuXGscbMfaaeis, address:145.79.227.45:4230): protocol: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->145.79.227.45:4230: use of closed network connection Jan 10 21:16:43.135438275 (869h47m43.00652132s) [noise=true , address=145.79.227.47:4266 , piece_id=FPZIU6JRPHIQHTR3REUGL7S4CHTAAOCRE4JOSPRGJLRNMG3UUJNQ, seg_pos={0 0}, upload=760991]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38582->145.79.227.47:4266: use of closed network connection Jan 10 21:16:43.135459575 (869h47m43.00654261s) [seg_pos={0 0}, upload=760991]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 10 21:16:43.374310077 (869h47m43.245393113s) [upload=760991]: Done uploading segments... Jan 10 21:16:43.374344443 (869h47m43.245427478s) [upload=760991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.240475499s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [..XX.X....X.X....XX.........X....X..........................] duration: 3.447119175s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:15:39 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.447145004s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [...XX.X....X.X....XX.........X....X.........................] duration: 3.634490307s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:15:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.634518291s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [X...XX.X....X.X....XX.........X....X........................] duration: 3.845333116s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:14:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.845354385s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [.X...XX.X....X.X....XX.........X....X.......................] duration: 3.686233469s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:14:05 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.686264629s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 11 out of 60 is failed (threshold=6) [X.X...XX.X....X.X....XX.........X....X......................] duration: 3.594018153s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:13:48 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.594041548s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 12 out of 60 is failed (threshold=6) [XX.X...XX.X....X.X....XX.........X....X.....................] duration: 3.516366093s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:13:10 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.516388776s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 12 out of 60 is failed (threshold=6) [.XX.X...XX.X....X.X....XX.........X....X....................] duration: 3.513708412s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:12:13 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.513732628s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 13 out of 60 is failed (threshold=6) [X.XX.X...XX.X....X.X....XX.........X....X...................] duration: 3.567998396s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:11:06 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.568025918s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 14 out of 60 is failed (threshold=6) [XX.XX.X...XX.X....X.X....XX.........X....X..................] duration: 3.657080409s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:10:30 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.657106631s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 15 out of 60 is failed (threshold=6) [XXX.XX.X...XX.X....X.X....XX.........X....X.................] duration: 3.608519976s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:09:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.608551195s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 15 out of 60 is failed (threshold=6) [.XXX.XX.X...XX.X....X.X....XX.........X....X................] duration: 3.731421687s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:08:34 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.731448238s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [X.XXX.XX.X...XX.X....X.X....XX.........X....X...............] duration: 3.48667822s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:08:15 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.48669323s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [XX.XXX.XX.X...XX.X....X.X....XX.........X....X..............] duration: 4.370669233s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:07:50 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.370691406s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [.XX.XXX.XX.X...XX.X....X.X....XX.........X....X.............] duration: 3.636661379s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:06:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.636689471s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X............] duration: 3.604491675s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:06:16 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.604521823s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X...........] duration: 3.617299142s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:05:10 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.617352503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 19 out of 60 is failed (threshold=6) [X.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X..........] duration: 3.531912995s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:04:09 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.531942042s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 20 out of 60 is failed (threshold=6) [XX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X.........] duration: 3.535229472s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:03:05 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.535256554s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 21 out of 60 is failed (threshold=6) [XXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X........] duration: 3.535841862s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:01:59 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.535894091s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X.......] duration: 3.520928431s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:01:04 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.520946085s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [.XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X......] duration: 3.731854502s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 21:00:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.731883667s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X.....] duration: 3.60849169s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 20:59:04 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.60851286s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 23 out of 60 is failed (threshold=6) [X..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X....] duration: 3.670209622s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 20:58:06 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.670234841s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [XX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X...] duration: 3.562526238s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 20:57:38 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.562552367s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [XXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X..] duration: 3.625673704s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 20:56:33 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.625689716s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 26 out of 60 is failed (threshold=6) [XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X.] duration: 5.001302917s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 20:55:40 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001329166s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 26 out of 60 is failed (threshold=6) [.XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....X] duration: 5.000864927s trace: https://jaeger.dev.storj.io/trace/493f577e1a9afb1 FAIL (2026-01-10 20:55:17 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00088295s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X....] duration: 3.586941595s trace: https://jaeger.dev.storj.io/trace/6069d6e8c918b3ff FAIL (2026-01-10 20:54:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.586967755s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 26 out of 60 is failed (threshold=6) [X..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X...] duration: 3.585455835s trace: https://jaeger.dev.storj.io/trace/6069d6e8c918b3ff FAIL (2026-01-10 20:53:43 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.58548946s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [XX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X..] duration: 3.52116309s trace: https://jaeger.dev.storj.io/trace/6069d6e8c918b3ff FAIL (2026-01-10 20:53:01 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.521187917s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X.] duration: 3.584526866s trace: https://jaeger.dev.storj.io/trace/6069d6e8c918b3ff FAIL (2026-01-10 20:52:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.584605066s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........X] duration: 5.001030475s trace: https://jaeger.dev.storj.io/trace/6069d6e8c918b3ff FAIL (2026-01-10 20:51:58 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00104955s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [X.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.........] duration: 3.68942416s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:51:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.689453818s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [XX.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX........] duration: 3.545185156s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:49:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.54521937s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 30 out of 60 is failed (threshold=6) [XXX.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.......] duration: 3.66694556s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:49:10 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.66696638s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 31 out of 60 is failed (threshold=6) [XXXX.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX......] duration: 3.595743404s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:48:22 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.595759213s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 32 out of 60 is failed (threshold=6) [XXXXX.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX.....] duration: 3.603501296s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:47:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.603527144s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 33 out of 60 is failed (threshold=6) [XXXXXX.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX....] duration: 3.496456214s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:47:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.496475141s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 34 out of 60 is failed (threshold=6) [XXXXXXX.XXX..XXXX..XXXX.X.XX.XXX.XX.X...XX.X....X.X....XX...] duration: 3.50331903s trace: https://jaeger.dev.storj.io/trace/3ac4da4dd76f20b9 FAIL (2026-01-10 20:46:26 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.503344529s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)