Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.101106036s FAIL (2025-10-06 00:06:59 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.101168415s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.408850907s FAIL (2025-10-06 00:06:16 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.408879622s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.404361089s FAIL (2025-10-06 00:05:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.404389232s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.322507452s FAIL (2025-10-06 00:04:41 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.322537219s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.254538356s FAIL (2025-10-06 00:04:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.254567931s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.311084991s FAIL (2025-10-06 00:03:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.311104348s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.041538515s FAIL (2025-10-06 00:02:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.041564033s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.239168759s FAIL (2025-10-06 00:01:07 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.239200591s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 7.137607073s FAIL (2025-10-06 00:00:15 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.137661676s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.178817619s FAIL (2025-10-05 23:59:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.178845963s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.909939758s FAIL (2025-10-05 23:58:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.910013559s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.419200253s FAIL (2025-10-05 23:58:41 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.419227093s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.386695849s FAIL (2025-10-05 23:57:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.386726758s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.1981922s FAIL (2025-10-05 23:57:51 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.198221104s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.374631055s FAIL (2025-10-05 23:57:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.374668267s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.603283056s FAIL (2025-10-05 23:56:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.603315968s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.47130004s FAIL (2025-10-05 23:56:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.471332s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.158339698s FAIL (2025-10-05 23:56:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.158369744s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.348155655s FAIL (2025-10-05 23:55:31 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.348185471s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.434122827s FAIL (2025-10-05 23:55:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.434159608s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.442265141s FAIL (2025-10-05 23:54:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.44229144s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.04930775s FAIL (2025-10-05 23:54:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.049340171s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.377980353s FAIL (2025-10-05 23:53:51 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.378004208s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.36398937s FAIL (2025-10-05 23:53:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.364030929s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.155988743s FAIL (2025-10-05 23:53:22 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.15602378s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.211452162s FAIL (2025-10-05 23:52:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.211480095s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.421773741s FAIL (2025-10-05 23:52:15 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.421801024s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.805782875s FAIL (2025-10-05 23:51:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.805814946s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.27015864s FAIL (2025-10-05 23:51:43 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.270190561s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.165185761s FAIL (2025-10-05 23:50:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.165213524s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.169325144s FAIL (2025-10-05 23:50:30 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.169355972s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.690594222s FAIL (2025-10-05 23:50:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.690643025s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.859418427s FAIL (2025-10-05 23:49:47 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.85944698s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.170446891s FAIL (2025-10-05 23:49:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.170479011s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00142459s FAIL (2025-10-05 23:49:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001449347s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 8.004137056s FAIL (2025-10-05 23:48:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.004173456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.950921459s FAIL (2025-10-05 23:47:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.950948913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 7.083246255s FAIL (2025-10-05 23:46:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.083279908s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.127116787s FAIL (2025-10-05 23:45:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.127154008s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.27492449s FAIL (2025-10-05 23:45:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.274979345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.166014095s FAIL (2025-10-05 23:44:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.166048719s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.076494142s FAIL (2025-10-05 23:43:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.076549977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.173709628s FAIL (2025-10-05 23:42:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.173743982s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.885172187s FAIL (2025-10-05 23:41:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.885209227s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.321237937s FAIL (2025-10-05 23:41:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.321263295s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.423963684s FAIL (2025-10-05 23:41:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.42399373s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.060516377s FAIL (2025-10-05 23:39:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.060549471s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.729980565s FAIL (2025-10-05 23:39:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.730044477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.974603926s FAIL (2025-10-05 23:38:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.974635084s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.163782034s FAIL (2025-10-05 23:38:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 5 23:15:12.666618443 (297h43m58.128710926s) [upload=262059]: Starting upload Oct 5 23:15:12.666708064 (297h43m58.128800537s) [upload=262059]: Done starting upload Oct 5 23:15:12.666765072 (297h43m58.128857565s) [upload=262059]: Uploading segments... Oct 5 23:15:12.666937019 (297h43m58.129029502s) [upload=262059]: Got next segment. Inline: false Oct 5 23:15:12.666976324 (297h43m58.129068807s) [upload=262059]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 5 23:15:12.666984791 (297h43m58.129077274s) [upload=262059]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 5 23:15:12.922193954 (297h43m58.384286437s) [seg_pos={0 0}, upload=262059]: Begin upload segment... Oct 5 23:15:12.922289196 (297h43m58.384381709s) [seg_pos={0 0}, upload=262059]: Done begin upload segment. Oct 5 23:15:12.922360271 (297h43m58.384452754s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922378486 (297h43m58.384470959s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.922852688 (297h43m58.384945161s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923214498 (297h43m58.385306981s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923538865 (297h43m58.385631348s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.923967762 (297h43m58.386060245s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924278924 (297h43m58.386371407s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.924697020 (297h43m58.386789503s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925052458 (297h43m58.387144941s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925464361 (297h43m58.387556845s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925485131 (297h43m58.387577614s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925792928 (297h43m58.387885411s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.925946119 (297h43m58.388038602s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926295344 (297h43m58.388387817s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.926645070 (297h43m58.388737543s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:12.927570382 (297h43m58.389662855s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Uploading piece... Oct 5 23:15:14.722074495 (297h44m0.184166969s) [upload=262059]: Next returned nil segment Oct 5 23:15:14.722100945 (297h44m0.184193428s) [upload=262059]: Waiting for error group managing segments... Oct 5 23:15:15.036812241 (297h44m0.498904714s) [noise=true , address=45.91.116.10:4081 , piece_id=O6HHQS7BJX5TEZN5OL3OSM3IIX4VX2KKCE4KQJ3CZYKDBZGXJJAA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.037311252 (297h44m0.499403725s) [noise=true , address=45.91.116.12:4227 , piece_id=GDR5OLR3ZFEM3AST7LGBXXGIA45FGXSXE5WOGRIDHKIBHD4PRNLQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.038474947 (297h44m0.50056743s) [noise=true , address=77.90.31.15:4699 , piece_id=LTNR3KBO7XSD53RURWWKNBJK2PVXTFTML7NLUSXKWXPOSJU2WX3A, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.065711180 (297h44m0.527803664s) [noise=true , address=45.91.116.6:4975 , piece_id=7CIMLJEIGSGMX6BHIPD5MZOICCT75IYALHWN2GEB2WQCBKKOJS6Q, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.072509606 (297h44m0.534602099s) [noise=true , address=77.90.31.21:4877 , piece_id=RD3QYLGQCLIXQUH2PEC6QCUE6H366BGTM6SMB7HUI5ILIGCNU4RA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.080945028 (297h44m0.543037511s) [noise=true , address=77.90.31.19:4843 , piece_id=6Y5GP2GNNM7MNKEM4QGR44WEMXDBABEJDWEG3OBJ4JRYX5IGLSPA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.095226102 (297h44m0.557318585s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2RIJVAQNLI75MK444EEPKFVIBE6CYSPJTUYZFAKOUZLX7OMX5NQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.112424591 (297h44m0.574517074s) [noise=true , address=45.91.116.8:4047 , piece_id=NBJ2RXJ6KXOF2ZOQEAGBWFMOO444XWNM5X4BXWZ4LCVRDAL2YYEA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.120512561 (297h44m0.582605045s) [noise=true , address=45.91.116.7:4010 , piece_id=NOO6UQH3WSLM6BU2QLEPHUULY7TPRGE4QSMUFQ6LIOSXSBIHUCWA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.263246472 (297h44m0.725338945s) [noise=true , address=77.90.31.18:4806 , piece_id=N3Z56LNTIRXOSLEEUMLI5ROYTDDJ6IR2ZK3JIZ4DVG6ASPSQQ6IQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.415636459 (297h44m0.877728932s) [noise=true , address=77.90.31.14:4663 , piece_id=OHEJUPJMA3EJ6FSCMYJRKECN4FWZ2XA2IDZUBQ63EC255Q3IYNNQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:15.417002901 (297h44m0.879095374s) [noise=true , address=45.91.116.4:4939 , piece_id=S7FOL55O2SJJQKEV6TWB2OXWJTQBBOV46V5SHNVA6RBLT4GP5RUQ, seg_pos={0 0}, upload=262059]: Done uploading piece. err: <nil> Oct 5 23:15:41.950461307 (297h44m27.41255379s) [noise=true , address=145.79.227.18:4005 , piece_id=4AYJPSHPP73JLOMAVCIUVO7ON3GN6NRNPAUMXO7E75CLTWW5C3HA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40926->145.79.227.18:4005: use of closed network connection Oct 5 23:15:41.950619337 (297h44m27.41271182s) [noise=true , address=145.79.227.15:4875 , piece_id=H3KHVH2ESEP4GVTDNKB3WR6JREZEGGBYLRITQXXM5WIAGFQV4EHA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33654->145.79.227.15:4875: use of closed network connection Oct 5 23:15:41.950719508 (297h44m27.412812001s) [noise=true , address=145.79.227.2:4114 , piece_id=FRCX4WFFI5BVHXUND4CNMWZND7VO4PY4AR5VGYFQBDMYZOHY4IJA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57132->145.79.227.2:4114: use of closed network connection Oct 5 23:15:41.950810011 (297h44m27.412902494s) [noise=true , address=145.79.227.14:4838 , piece_id=MVKBB52K5ZYWZYCMUZ7UO7GVX7A2ZOMH2ZLT5UHG5GHDXPWTPPOA, seg_pos={0 0}, upload=262059]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34012->145.79.227.14:4838: use of closed network connection Oct 5 23:15:41.950839317 (297h44m27.41293179s) [seg_pos={0 0}, upload=262059]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 5 23:15:42.203958951 (297h44m27.666051444s) [upload=262059]: Done uploading segments... Oct 5 23:15:42.203991713 (297h44m27.666084196s) [upload=262059]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.163805067s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)