Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.493412338s FAIL (2025-10-20 20:21:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 20 20:20:22.556080189 (654h49m8.018172672s) [upload=576103]: Starting upload Oct 20 20:20:22.556174328 (654h49m8.018266812s) [upload=576103]: Done starting upload Oct 20 20:20:22.556397533 (654h49m8.018490026s) [upload=576103]: Uploading segments... Oct 20 20:20:22.556567827 (654h49m8.018660311s) [upload=576103]: Got next segment. Inline: false Oct 20 20:20:22.556730538 (654h49m8.018823031s) [upload=576103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 20:20:22.556763350 (654h49m8.018855833s) [upload=576103]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 20:20:22.812162149 (654h49m8.274254632s) [seg_pos={0 0}, upload=576103]: Begin upload segment... Oct 20 20:20:22.812236560 (654h49m8.274329043s) [seg_pos={0 0}, upload=576103]: Done begin upload segment. Oct 20 20:20:22.812273581 (654h49m8.274366074s) [noise=true , address=145.79.227.18:4007 , piece_id=RX43ZIKMAIOW4Q6UBFSWHHLQUKTJKUWUKWXFD6X7MILVMGTEZOJQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.812309659 (654h49m8.274402152s) [noise=true , address=145.79.227.19:4079 , piece_id=CIGALYB627LSTS6PCU4MSDAJQMCPVSQE456ZVCHQ7KHZC26D4HSA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.813061832 (654h49m8.275154325s) [noise=true , address=145.79.227.4:4186 , piece_id=BN4IEVWCAJHIHGV5DMDRJCAJSB3PW2KHQQT2A64TGOUWHJMCKQGQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.813575600 (654h49m8.275668083s) [noise=true , address=45.91.116.13:4155 , piece_id=XQLOEZQXOY7ZCWGAXKZLTQV3FXY2HVQP7YJD57V2MNVEFKJZICRQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.814122902 (654h49m8.276215385s) [noise=true , address=145.79.227.11:4731 , piece_id=LIVVWJK5ZZ647EBTC52WOHXEUNWHOXKWNKUCAID2M2I6VOB4ZOAQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.814579291 (654h49m8.276671774s) [noise=true , address=145.79.227.17:4971 , piece_id=TNMRPAK5K6G7XF3T2C23ADIISV4STVX5J6C7BJQSMR5NSOVINIZQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.815074153 (654h49m8.277166636s) [noise=true , address=145.79.227.17:4970 , piece_id=NEI2COJTCPP363GAI2FTICYH6GCX43CE5XDAXWVLVUA7GBLWH5HQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.816220195 (654h49m8.278312678s) [noise=true , address=145.79.227.2:4113 , piece_id=YOFNTWXBADVUBKQQACJMGSS4VCUQLBRBB5QNQKLZPFS25GHKDJNQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.817767853 (654h49m8.279860336s) [noise=true , address=77.90.31.10:4517 , piece_id=RYLF6GGXT3E54MJ6HXFLY4EJNXY5O7HS3IYWVNWVGWCDPFVGI7LQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.819997118 (654h49m8.282089591s) [noise=true , address=45.91.116.4:4940 , piece_id=HXORS3GQ3RFGCOIUWJEYZHEOGCH5MTPFM6CYL45SIEZRUCDWXV5Q, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.822075064 (654h49m8.284167547s) [noise=true , address=145.79.227.19:4077 , piece_id=BSX3DG3BUDTUDIJEUUD23BYMGCAPUFXBODH6R44SSHWDTVCWYIGA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.823471253 (654h49m8.285563736s) [noise=true , address=45.91.116.4:4938 , piece_id=DZ7TFMV3ZIUCHVTG72WRKTS6BNIFSYV6PCRKBMUGAH5XHWAM4Z4A, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.825088132 (654h49m8.287180615s) [noise=true , address=77.90.31.14:4661 , piece_id=XJA4QSTMU6PZWJIBYPZDR2ZSG7SWHR5IK6NRZDYYTYTRXD5E67KA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.826407254 (654h49m8.288499727s) [noise=true , address=77.90.31.16:4733 , piece_id=CWSZMO23STVTB6AEJBDMTKJJGJNPQVUPGRI3DGQS7L2RIBTG5LXQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.828151625 (654h49m8.290244108s) [noise=true , address=145.79.227.7:4293 , piece_id=5U7ETGZ735RCZMRXSG2BHHAMSVWHCOHKFTN5RXEKTN73PUF3G6DQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.830155461 (654h49m8.292247954s) [noise=true , address=77.90.31.17:4771 , piece_id=MAWW3A53MLZLNLM3MFAGMT5DRD3YMYYQXREIAA46JCN4JEEILYGA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:24.535313550 (654h49m9.997406033s) [upload=576103]: Next returned nil segment Oct 20 20:20:24.535340031 (654h49m9.997432514s) [upload=576103]: Waiting for error group managing segments... Oct 20 20:20:24.821694104 (654h49m10.283786587s) [noise=true , address=145.79.227.17:4971 , piece_id=TNMRPAK5K6G7XF3T2C23ADIISV4STVX5J6C7BJQSMR5NSOVINIZQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.834691713 (654h49m10.296784196s) [noise=true , address=145.79.227.17:4970 , piece_id=NEI2COJTCPP363GAI2FTICYH6GCX43CE5XDAXWVLVUA7GBLWH5HQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.835933959 (654h49m10.298026452s) [noise=true , address=145.79.227.19:4079 , piece_id=CIGALYB627LSTS6PCU4MSDAJQMCPVSQE456ZVCHQ7KHZC26D4HSA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.842978072 (654h49m10.305070555s) [noise=true , address=145.79.227.18:4007 , piece_id=RX43ZIKMAIOW4Q6UBFSWHHLQUKTJKUWUKWXFD6X7MILVMGTEZOJQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.854156909 (654h49m10.316249392s) [noise=true , address=145.79.227.4:4186 , piece_id=BN4IEVWCAJHIHGV5DMDRJCAJSB3PW2KHQQT2A64TGOUWHJMCKQGQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.856119556 (654h49m10.318212039s) [noise=true , address=145.79.227.2:4113 , piece_id=YOFNTWXBADVUBKQQACJMGSS4VCUQLBRBB5QNQKLZPFS25GHKDJNQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.875100009 (654h49m10.337192492s) [noise=true , address=145.79.227.11:4731 , piece_id=LIVVWJK5ZZ647EBTC52WOHXEUNWHOXKWNKUCAID2M2I6VOB4ZOAQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.924163254 (654h49m10.386255737s) [noise=true , address=77.90.31.14:4661 , piece_id=XJA4QSTMU6PZWJIBYPZDR2ZSG7SWHR5IK6NRZDYYTYTRXD5E67KA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.942163912 (654h49m10.404256395s) [noise=true , address=77.90.31.16:4733 , piece_id=CWSZMO23STVTB6AEJBDMTKJJGJNPQVUPGRI3DGQS7L2RIBTG5LXQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.965494587 (654h49m10.42758706s) [noise=true , address=45.91.116.4:4940 , piece_id=HXORS3GQ3RFGCOIUWJEYZHEOGCH5MTPFM6CYL45SIEZRUCDWXV5Q, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.969679086 (654h49m10.431771569s) [noise=true , address=77.90.31.17:4771 , piece_id=MAWW3A53MLZLNLM3MFAGMT5DRD3YMYYQXREIAA46JCN4JEEILYGA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.982465964 (654h49m10.444558437s) [noise=true , address=145.79.227.7:4293 , piece_id=5U7ETGZ735RCZMRXSG2BHHAMSVWHCOHKFTN5RXEKTN73PUF3G6DQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.986089063 (654h49m10.448181546s) [noise=true , address=77.90.31.10:4517 , piece_id=RYLF6GGXT3E54MJ6HXFLY4EJNXY5O7HS3IYWVNWVGWCDPFVGI7LQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.986126986 (654h49m10.448219479s) [seg_pos={0 0}, upload=576103]: Segment reached optimal threshold of 13 pieces. Oct 20 20:20:24.986327046 (654h49m10.448419539s) [noise=true , address=45.91.116.13:4155 , piece_id=XQLOEZQXOY7ZCWGAXKZLTQV3FXY2HVQP7YJD57V2MNVEFKJZICRQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Oct 20 20:20:24.986340943 (654h49m10.448433426s) [noise=true , address=45.91.116.13:4155 , piece_id=XQLOEZQXOY7ZCWGAXKZLTQV3FXY2HVQP7YJD57V2MNVEFKJZICRQ, seg_pos={0 0}, upload=576103]: Long tail cancellation due to optimal threshold Oct 20 20:20:24.986349860 (654h49m10.448442353s) [noise=true , address=45.91.116.4:4938 , piece_id=DZ7TFMV3ZIUCHVTG72WRKTS6BNIFSYV6PCRKBMUGAH5XHWAM4Z4A, seg_pos={0 0}, upload=576103]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Oct 20 20:20:24.986358987 (654h49m10.44845147s) [noise=true , address=45.91.116.4:4938 , piece_id=DZ7TFMV3ZIUCHVTG72WRKTS6BNIFSYV6PCRKBMUGAH5XHWAM4Z4A, seg_pos={0 0}, upload=576103]: Long tail cancellation due to optimal threshold Oct 20 20:20:24.986714083 (654h49m10.448806566s) [noise=true , address=145.79.227.19:4077 , piece_id=BSX3DG3BUDTUDIJEUUD23BYMGCAPUFXBODH6R44SSHWDTVCWYIGA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: write tcp 188.166.225.170:52586->145.79.227.19:4077: use of closed network connection; write tcp 188.166.225.170:52586->145.79.227.19:4077: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52586->145.79.227.19:4077: use of closed network connection Oct 20 20:20:24.986726728 (654h49m10.448819211s) [noise=true , address=145.79.227.19:4077 , piece_id=BSX3DG3BUDTUDIJEUUD23BYMGCAPUFXBODH6R44SSHWDTVCWYIGA, seg_pos={0 0}, upload=576103]: Long tail cancellation due to optimal threshold Oct 20 20:20:24.986748429 (654h49m10.448840912s) [seg_pos={0 0}, upload=576103]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 20 20:20:24.986839031 (654h49m10.448931515s) [upload=576103]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 20 20:20:24.986845153 (654h49m10.448937636s) [upload=576103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 20 20:20:25.309524076 (654h49m10.771616559s) [upload=576103]: Done uploading segments... Oct 20 20:20:25.309561317 (654h49m10.7716538s) [upload=576103]: Upload finished. err: <nil> Duration: 7.493501969s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002859252s FAIL (2025-10-20 20:20:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 20 20:20:22.556080189 (654h49m8.018172672s) [upload=576103]: Starting upload Oct 20 20:20:22.556174328 (654h49m8.018266812s) [upload=576103]: Done starting upload Oct 20 20:20:22.556397533 (654h49m8.018490026s) [upload=576103]: Uploading segments... Oct 20 20:20:22.556567827 (654h49m8.018660311s) [upload=576103]: Got next segment. Inline: false Oct 20 20:20:22.556730538 (654h49m8.018823031s) [upload=576103]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 20:20:22.556763350 (654h49m8.018855833s) [upload=576103]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 20:20:22.812162149 (654h49m8.274254632s) [seg_pos={0 0}, upload=576103]: Begin upload segment... Oct 20 20:20:22.812236560 (654h49m8.274329043s) [seg_pos={0 0}, upload=576103]: Done begin upload segment. Oct 20 20:20:22.812273581 (654h49m8.274366074s) [noise=true , address=145.79.227.18:4007 , piece_id=RX43ZIKMAIOW4Q6UBFSWHHLQUKTJKUWUKWXFD6X7MILVMGTEZOJQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.812309659 (654h49m8.274402152s) [noise=true , address=145.79.227.19:4079 , piece_id=CIGALYB627LSTS6PCU4MSDAJQMCPVSQE456ZVCHQ7KHZC26D4HSA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.813061832 (654h49m8.275154325s) [noise=true , address=145.79.227.4:4186 , piece_id=BN4IEVWCAJHIHGV5DMDRJCAJSB3PW2KHQQT2A64TGOUWHJMCKQGQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.813575600 (654h49m8.275668083s) [noise=true , address=45.91.116.13:4155 , piece_id=XQLOEZQXOY7ZCWGAXKZLTQV3FXY2HVQP7YJD57V2MNVEFKJZICRQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.814122902 (654h49m8.276215385s) [noise=true , address=145.79.227.11:4731 , piece_id=LIVVWJK5ZZ647EBTC52WOHXEUNWHOXKWNKUCAID2M2I6VOB4ZOAQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.814579291 (654h49m8.276671774s) [noise=true , address=145.79.227.17:4971 , piece_id=TNMRPAK5K6G7XF3T2C23ADIISV4STVX5J6C7BJQSMR5NSOVINIZQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.815074153 (654h49m8.277166636s) [noise=true , address=145.79.227.17:4970 , piece_id=NEI2COJTCPP363GAI2FTICYH6GCX43CE5XDAXWVLVUA7GBLWH5HQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.816220195 (654h49m8.278312678s) [noise=true , address=145.79.227.2:4113 , piece_id=YOFNTWXBADVUBKQQACJMGSS4VCUQLBRBB5QNQKLZPFS25GHKDJNQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.817767853 (654h49m8.279860336s) [noise=true , address=77.90.31.10:4517 , piece_id=RYLF6GGXT3E54MJ6HXFLY4EJNXY5O7HS3IYWVNWVGWCDPFVGI7LQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.819997118 (654h49m8.282089591s) [noise=true , address=45.91.116.4:4940 , piece_id=HXORS3GQ3RFGCOIUWJEYZHEOGCH5MTPFM6CYL45SIEZRUCDWXV5Q, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.822075064 (654h49m8.284167547s) [noise=true , address=145.79.227.19:4077 , piece_id=BSX3DG3BUDTUDIJEUUD23BYMGCAPUFXBODH6R44SSHWDTVCWYIGA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.823471253 (654h49m8.285563736s) [noise=true , address=45.91.116.4:4938 , piece_id=DZ7TFMV3ZIUCHVTG72WRKTS6BNIFSYV6PCRKBMUGAH5XHWAM4Z4A, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.825088132 (654h49m8.287180615s) [noise=true , address=77.90.31.14:4661 , piece_id=XJA4QSTMU6PZWJIBYPZDR2ZSG7SWHR5IK6NRZDYYTYTRXD5E67KA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.826407254 (654h49m8.288499727s) [noise=true , address=77.90.31.16:4733 , piece_id=CWSZMO23STVTB6AEJBDMTKJJGJNPQVUPGRI3DGQS7L2RIBTG5LXQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.828151625 (654h49m8.290244108s) [noise=true , address=145.79.227.7:4293 , piece_id=5U7ETGZ735RCZMRXSG2BHHAMSVWHCOHKFTN5RXEKTN73PUF3G6DQ, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:22.830155461 (654h49m8.292247954s) [noise=true , address=77.90.31.17:4771 , piece_id=MAWW3A53MLZLNLM3MFAGMT5DRD3YMYYQXREIAA46JCN4JEEILYGA, seg_pos={0 0}, upload=576103]: Uploading piece... Oct 20 20:20:24.535313550 (654h49m9.997406033s) [upload=576103]: Next returned nil segment Oct 20 20:20:24.535340031 (654h49m9.997432514s) [upload=576103]: Waiting for error group managing segments... Oct 20 20:20:24.821694104 (654h49m10.283786587s) [noise=true , address=145.79.227.17:4971 , piece_id=TNMRPAK5K6G7XF3T2C23ADIISV4STVX5J6C7BJQSMR5NSOVINIZQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.834691713 (654h49m10.296784196s) [noise=true , address=145.79.227.17:4970 , piece_id=NEI2COJTCPP363GAI2FTICYH6GCX43CE5XDAXWVLVUA7GBLWH5HQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.835933959 (654h49m10.298026452s) [noise=true , address=145.79.227.19:4079 , piece_id=CIGALYB627LSTS6PCU4MSDAJQMCPVSQE456ZVCHQ7KHZC26D4HSA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.842978072 (654h49m10.305070555s) [noise=true , address=145.79.227.18:4007 , piece_id=RX43ZIKMAIOW4Q6UBFSWHHLQUKTJKUWUKWXFD6X7MILVMGTEZOJQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.854156909 (654h49m10.316249392s) [noise=true , address=145.79.227.4:4186 , piece_id=BN4IEVWCAJHIHGV5DMDRJCAJSB3PW2KHQQT2A64TGOUWHJMCKQGQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.856119556 (654h49m10.318212039s) [noise=true , address=145.79.227.2:4113 , piece_id=YOFNTWXBADVUBKQQACJMGSS4VCUQLBRBB5QNQKLZPFS25GHKDJNQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.875100009 (654h49m10.337192492s) [noise=true , address=145.79.227.11:4731 , piece_id=LIVVWJK5ZZ647EBTC52WOHXEUNWHOXKWNKUCAID2M2I6VOB4ZOAQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.924163254 (654h49m10.386255737s) [noise=true , address=77.90.31.14:4661 , piece_id=XJA4QSTMU6PZWJIBYPZDR2ZSG7SWHR5IK6NRZDYYTYTRXD5E67KA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.942163912 (654h49m10.404256395s) [noise=true , address=77.90.31.16:4733 , piece_id=CWSZMO23STVTB6AEJBDMTKJJGJNPQVUPGRI3DGQS7L2RIBTG5LXQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.965494587 (654h49m10.42758706s) [noise=true , address=45.91.116.4:4940 , piece_id=HXORS3GQ3RFGCOIUWJEYZHEOGCH5MTPFM6CYL45SIEZRUCDWXV5Q, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.969679086 (654h49m10.431771569s) [noise=true , address=77.90.31.17:4771 , piece_id=MAWW3A53MLZLNLM3MFAGMT5DRD3YMYYQXREIAA46JCN4JEEILYGA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.982465964 (654h49m10.444558437s) [noise=true , address=145.79.227.7:4293 , piece_id=5U7ETGZ735RCZMRXSG2BHHAMSVWHCOHKFTN5RXEKTN73PUF3G6DQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.986089063 (654h49m10.448181546s) [noise=true , address=77.90.31.10:4517 , piece_id=RYLF6GGXT3E54MJ6HXFLY4EJNXY5O7HS3IYWVNWVGWCDPFVGI7LQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: <nil> Oct 20 20:20:24.986126986 (654h49m10.448219479s) [seg_pos={0 0}, upload=576103]: Segment reached optimal threshold of 13 pieces. Oct 20 20:20:24.986327046 (654h49m10.448419539s) [noise=true , address=45.91.116.13:4155 , piece_id=XQLOEZQXOY7ZCWGAXKZLTQV3FXY2HVQP7YJD57V2MNVEFKJZICRQ, seg_pos={0 0}, upload=576103]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Oct 20 20:20:24.986340943 (654h49m10.448433426s) [noise=true , address=45.91.116.13:4155 , piece_id=XQLOEZQXOY7ZCWGAXKZLTQV3FXY2HVQP7YJD57V2MNVEFKJZICRQ, seg_pos={0 0}, upload=576103]: Long tail cancellation due to optimal threshold Oct 20 20:20:24.986349860 (654h49m10.448442353s) [noise=true , address=45.91.116.4:4938 , piece_id=DZ7TFMV3ZIUCHVTG72WRKTS6BNIFSYV6PCRKBMUGAH5XHWAM4Z4A, seg_pos={0 0}, upload=576103]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Oct 20 20:20:24.986358987 (654h49m10.44845147s) [noise=true , address=45.91.116.4:4938 , piece_id=DZ7TFMV3ZIUCHVTG72WRKTS6BNIFSYV6PCRKBMUGAH5XHWAM4Z4A, seg_pos={0 0}, upload=576103]: Long tail cancellation due to optimal threshold Oct 20 20:20:24.986714083 (654h49m10.448806566s) [noise=true , address=145.79.227.19:4077 , piece_id=BSX3DG3BUDTUDIJEUUD23BYMGCAPUFXBODH6R44SSHWDTVCWYIGA, seg_pos={0 0}, upload=576103]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: write tcp 188.166.225.170:52586->145.79.227.19:4077: use of closed network connection; write tcp 188.166.225.170:52586->145.79.227.19:4077: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52586->145.79.227.19:4077: use of closed network connection Oct 20 20:20:24.986726728 (654h49m10.448819211s) [noise=true , address=145.79.227.19:4077 , piece_id=BSX3DG3BUDTUDIJEUUD23BYMGCAPUFXBODH6R44SSHWDTVCWYIGA, seg_pos={0 0}, upload=576103]: Long tail cancellation due to optimal threshold Oct 20 20:20:24.986748429 (654h49m10.448840912s) [seg_pos={0 0}, upload=576103]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 20 20:20:24.986839031 (654h49m10.448931515s) [upload=576103]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 20 20:20:24.986845153 (654h49m10.448937636s) [upload=576103]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 20 20:20:25.309524076 (654h49m10.771616559s) [upload=576103]: Done uploading segments... Oct 20 20:20:25.309561317 (654h49m10.7716538s) [upload=576103]: Upload finished. err: <nil> Duration: 30.002882477s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 191.649216ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:20:38 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 191.702518ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 192.612611ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:19:32 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 192.638912ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 208.228969ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:18:30 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 208.254428ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 169.945369ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:17:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 169.97688ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 180.176492ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:17:05 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 180.218572ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 172.070354ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:16:28 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 172.168973ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 177.406146ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:15:16 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 177.548016ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 169.798508ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:14:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 169.828245ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 170.854309ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:13:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 170.88078ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 173.929955ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:13:14 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 174.002754ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 174.789844ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:12:50 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 174.817818ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 185.216137ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:12:30 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 185.315757ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 177.285516ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:12:07 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 177.37184ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 198.601275ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:10:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 198.638657ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 194.627228ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:09:40 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 194.658708ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 196.583563ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:08:43 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 196.616066ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 207.679692ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:07:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 207.724979ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 200.10577ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:07:17 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 200.250466ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 228.408935ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:06:05 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 228.444693ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 200.078799ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:05:11 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 200.109558ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 193.745768ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:04:41 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 193.772399ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 188.420649ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:03:33 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 188.46259ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 195.896515ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:02:30 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 195.922234ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 197.255914ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:01:47 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 197.284848ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 244.048195ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:01:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 244.081778ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 1.449820789s target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 20:00:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 1.449862167s edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 203.567461ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:59:06 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 203.650549ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 185.358257ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:58:07 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 185.397252ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 181.674445ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:57:40 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 181.724661ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 197.736206ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:56:32 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 197.799408ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 197.204435ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:55:34 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 197.237899ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 199.609635ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:55:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 199.629243ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 30.001045193s target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-20 19:54:35 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.001081112s edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.013789561s FAIL (2025-10-20 17:54:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.013824619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.010995922s FAIL (2025-10-20 17:54:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.011030508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.914157025s FAIL (2025-10-20 17:53:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.914179708s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.36284442s FAIL (2025-10-20 17:53:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.362872915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.935255129s FAIL (2025-10-20 17:52:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.935293181s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.121438085s FAIL (2025-10-20 17:51:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.121468823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.160878798s FAIL (2025-10-20 17:51:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.160919355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.996480342s FAIL (2025-10-20 17:50:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.996509528s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.479386424s FAIL (2025-10-20 17:50:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.479422373s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.30398584s FAIL (2025-10-20 17:50:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.304020575s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.28754912s FAIL (2025-10-20 17:49:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.287594918s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.418624362s FAIL (2025-10-20 17:49:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.41867629s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.399513384s FAIL (2025-10-20 17:48:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.399550164s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.056089423s FAIL (2025-10-20 17:48:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== e starting upload Oct 20 17:07:22.290572384 (651h36m7.752664867s) [upload=573243]: Uploading segments... Oct 20 17:07:22.290611870 (651h36m7.752704353s) [upload=573243]: Got next segment. Inline: false Oct 20 17:07:22.290645023 (651h36m7.752737507s) [upload=573243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 20 17:07:22.290651015 (651h36m7.752743498s) [upload=573243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 20 17:07:23.663328653 (651h36m9.125421166s) [seg_pos={0 0}, upload=573243]: Begin upload segment... Oct 20 17:07:23.663406633 (651h36m9.125499106s) [seg_pos={0 0}, upload=573243]: Done begin upload segment. Oct 20 17:07:23.663442671 (651h36m9.125535144s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664168974 (651h36m9.126261447s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.664669607 (651h36m9.12676208s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665187343 (651h36m9.127279826s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665511901 (651h36m9.127604374s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.665899328 (651h36m9.127991801s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666282578 (651h36m9.128375051s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.666636282 (651h36m9.128728755s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667140843 (651h36m9.129233316s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.667918994 (651h36m9.130011477s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668549374 (651h36m9.130641858s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.668950689 (651h36m9.131043172s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669455600 (651h36m9.131548083s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.669546133 (651h36m9.131638616s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670135064 (651h36m9.132227537s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:23.670524335 (651h36m9.132616818s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Uploading piece... Oct 20 17:07:39.081587577 (651h36m24.54368006s) [upload=573243]: Next returned nil segment Oct 20 17:07:39.081611963 (651h36m24.543704437s) [upload=573243]: Waiting for error group managing segments... Oct 20 17:07:41.933983393 (651h36m27.396075876s) [noise=true , address=77.90.31.14:4662 , piece_id=MGTMNXXDRMJVWMGAPGZ7TITNKV34GIJNNCNF6HELMG2QWISADMMA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:46.418908042 (651h36m31.881000525s) [noise=true , address=77.90.31.21:4879 , piece_id=NE32VKU5Y5WKDCN5Q4Z4426MYFTSW2JD65G52WHQWSH5AQPXJOOA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:47.945682340 (651h36m33.407774813s) [noise=true , address=145.79.227.19:4077 , piece_id=2HLR5JFCMCE3TVP4JSWB6SDC2H7BHP7W42XHJEILAZGILDBEQ6PA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:48.081389863 (651h36m33.543482347s) [noise=true , address=145.79.227.6:4257 , piece_id=EMQGGIF2NCJ746YZ4D5FZRFFVZLI6JNUNNJAIKVOY3DTAELSRP3Q, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.014359976 (651h36m34.476452459s) [noise=true , address=145.79.227.14:4840 , piece_id=SUSZYHF7CJ7QEJD4O5A4OMSOHRFH55PXJJT2FTZQV3CUGJCTIUFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.553746602 (651h36m35.015839085s) [noise=true , address=77.90.31.17:4771 , piece_id=M6QPWFXO5K3VJGF7OTIBRSJN7EV47VRHBKFL5DMPFADQHYCO7U4A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.645207982 (651h36m35.107300455s) [noise=true , address=77.90.31.16:4735 , piece_id=ZYMRZMPD6BRQ6YBB7UBKSV6JWNAZNWIOZYXZQE5MM7IJDKMPP4WQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:49.962342927 (651h36m35.4244354s) [noise=true , address=145.79.227.11:4731 , piece_id=YE6RICRM4KSUSCTL7F2RX3FMHSFRS2ELD7QKACEB3Q5DW6K5TS6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.102101054 (651h36m35.564193527s) [noise=true , address=77.90.31.21:4878 , piece_id=ZYGWQNJ6ISGSU2BASHWWAHZYWZ2D5US7EQT5AVSFL552XREVWKRQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:50.656974057 (651h36m36.11906654s) [noise=true , address=45.91.116.4:4938 , piece_id=GVCINZLOFHWH46PW3YRZBZKJHIXOPAEM5B5LMVRX2256L225UWAQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: <nil> Oct 20 17:07:51.421289780 (651h36m36.883382263s) [noise=true , address=45.91.116.11:4119 , piece_id=CUCKNF4W7UAYA4MZVLGKPALFIYXAIAE5LEJ437GFLP6V2CANMXPA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421341999 (651h36m36.883434482s) [noise=true , address=45.91.116.10:4082 , piece_id=7SIT5DWSJ7XCSCPIUJ3ZLXUZB5G2W455JVIFMLXOQOL73BV34A6A, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44202->45.91.116.10:4082: use of closed network connection Oct 20 17:07:51.421412643 (651h36m36.883505116s) [noise=true , address=77.90.31.18:4807 , piece_id=GU6WYQZP6A52MXJQJ4WRK25X5RJWVM3UR3GAH2QA74AHQ43CL6DA, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Oct 20 17:07:51.421447399 (651h36m36.883539883s) [noise=true , address=145.79.227.18:4005 , piece_id=GCKQ2JVL7UY7SVCQVI2ZOUHOOC3WEWXIWOXYRXHMLMI7AZLXOJZQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32922->145.79.227.18:4005: use of closed network connection Oct 20 17:07:51.421618105 (651h36m36.883710588s) [noise=true , address=77.90.31.16:4734 , piece_id=MKUE6LYTA24S6QO3GF7ETNDYGYKGJGECVIRQPXLXC4XNA3ITG5BQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51468->77.90.31.16:4734: use of closed network connection Oct 20 17:07:51.421657980 (651h36m36.883750453s) [noise=true , address=77.90.31.10:4517 , piece_id=DRQWHUW5MZHELNFO3SMFZSYFZGFP7EU52XYEPVZGHKGPTEAFCHFQ, seg_pos={0 0}, upload=573243]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.10:4517: use of closed network connection Oct 20 17:07:51.421709749 (651h36m36.883802233s) [seg_pos={0 0}, upload=573243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 20 17:07:51.670217656 (651h36m37.132310159s) [upload=573243]: Done uploading segments... Oct 20 17:07:51.670300122 (651h36m37.132392615s) [upload=573243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.056127546s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.356642589s FAIL (2025-10-20 17:48:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.356670592s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)