Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 9.65717911s FAIL (2025-09-14 14:56:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 9.657207545s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 7.757357185s FAIL (2025-09-14 14:56:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.75739111s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.613323458s FAIL (2025-09-14 14:56:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.613347554s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 9.174436574s FAIL (2025-09-14 14:55:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 9.174466412s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.942994479s FAIL (2025-09-14 14:55:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.94302072s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 8.086030316s FAIL (2025-09-14 14:54:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 8.086055043s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 7.230243534s FAIL (2025-09-14 14:54:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.230291897s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 14.063035092s FAIL (2025-09-14 14:54:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 14.063062904s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.877134625s FAIL (2025-09-14 14:54:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.877161506s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 7.558036248s FAIL (2025-09-14 14:53:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.558064843s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.057682743s FAIL (2025-09-14 14:53:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.057699726s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 7.448945352s FAIL (2025-09-14 14:53:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.448968277s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.762322978s FAIL (2025-09-14 14:53:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.762376961s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 12.257166329s FAIL (2025-09-14 14:52:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 12.257196768s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.906255236s FAIL (2025-09-14 14:52:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.906916241s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.77088992s FAIL (2025-09-14 14:51:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.770928123s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 7.460543526s FAIL (2025-09-14 14:51:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.460563323s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.684466507s FAIL (2025-09-14 14:50:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.684516041s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 7.834000964s FAIL (2025-09-14 14:50:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.834027865s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.500070926s FAIL (2025-09-14 14:50:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.500098779s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.624141909s FAIL (2025-09-14 14:49:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.624191996s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.890916698s FAIL (2025-09-14 14:49:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 6.890956725s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.814303412s FAIL (2025-09-14 14:49:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.814331106s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.774927042s FAIL (2025-09-14 14:48:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.774949547s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 11.891317486s FAIL (2025-09-14 14:48:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 11.891347154s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.601773599s FAIL (2025-09-14 14:48:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.601803969s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 13.409829046s FAIL (2025-09-14 14:47:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 13.409847581s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 7.080452441s FAIL (2025-09-14 14:47:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.080475927s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 7.50405003s FAIL (2025-09-14 14:47:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.504074137s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 8.745052025s FAIL (2025-09-14 14:47:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 8.745098705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 13.378355269s FAIL (2025-09-14 14:46:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 13.378400826s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.972113902s FAIL (2025-09-14 14:46:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.972134481s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.832654391s FAIL (2025-09-14 14:46:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.832675961s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 7.894854122s FAIL (2025-09-14 14:45:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.894879892s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 7.449950383s FAIL (2025-09-14 14:45:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.450219588s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 7.08883336s FAIL (2025-09-14 14:44:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.088878125s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 8.682131336s FAIL (2025-09-14 14:44:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 8.682160071s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.909890588s FAIL (2025-09-14 14:44:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.90990789s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 8.556597585s FAIL (2025-09-14 14:43:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 8.556624756s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.806629439s FAIL (2025-09-14 14:43:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.806652354s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 7.839054147s FAIL (2025-09-14 14:43:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.83908664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 7.939828056s FAIL (2025-09-14 14:42:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.939847093s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.572526982s FAIL (2025-09-14 14:42:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.5725523s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.923264075s FAIL (2025-09-14 14:42:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.923982481s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 7.169973126s FAIL (2025-09-14 14:42:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 7.170006399s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 7.326179481s FAIL (2025-09-14 14:41:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.326203297s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 9.429123337s FAIL (2025-09-14 14:41:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 9.429149568s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.374495582s FAIL (2025-09-14 14:40:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.37452054s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 8.969305289s FAIL (2025-09-14 14:40:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 14 14:33:49.548887912 (193h5m25.588461032s) [upload=169699]: Starting upload Sep 14 14:33:49.548946624 (193h5m25.588519734s) [upload=169699]: Done starting upload Sep 14 14:33:49.549083356 (193h5m25.588656476s) [upload=169699]: Uploading segments... Sep 14 14:33:49.549183488 (193h5m25.588756618s) [upload=169699]: Got next segment. Inline: false Sep 14 14:33:49.549218695 (193h5m25.588791815s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 14 14:33:49.549223734 (193h5m25.588796854s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 14 14:33:50.936537330 (193h5m26.97611045s) [seg_pos={0 0}, upload=169699]: Begin upload segment... Sep 14 14:33:50.936582296 (193h5m26.976155416s) [seg_pos={0 0}, upload=169699]: Done begin upload segment. Sep 14 14:33:50.936604238 (193h5m26.976177358s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936652431 (193h5m26.976225551s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.936938217 (193h5m26.976511337s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937124093 (193h5m26.976697213s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937152497 (193h5m26.976725617s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937413867 (193h5m26.976986987s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937483711 (193h5m26.977056831s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937634891 (193h5m26.977208011s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937882875 (193h5m26.977455995s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.937892063 (193h5m26.977465183s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938108958 (193h5m26.977682078s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938278332 (193h5m26.977851452s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938401979 (193h5m26.977975099s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.938690290 (193h5m26.97826341s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939026153 (193h5m26.978599273s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:50.939858876 (193h5m26.979432006s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Uploading piece... Sep 14 14:33:55.764154731 (193h5m31.803727851s) [upload=169699]: Next returned nil segment Sep 14 14:33:55.764172495 (193h5m31.803745615s) [upload=169699]: Waiting for error group managing segments... Sep 14 14:33:57.261072911 (193h5m33.300646031s) [noise=true , address=45.91.116.7:4011 , piece_id=Q364XOAYC2OKYNURSWLT5JQIID3RPCFCY24QPY5OYVJR7RTI5E7A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.544293605 (193h5m33.583866725s) [noise=true , address=77.90.31.11:4553 , piece_id=L7BBTXT5YV5EFZWFBHTYFITALJYQGQHGFV5IVCJQZYFOVXZLBDFA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.548268682 (193h5m33.587841802s) [noise=true , address=77.90.31.18:4807 , piece_id=OQGKYC37IVAB55NDX5OQKGDZKTRDTDWOB3AECV3JP3E2QEGVSHZQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.569822607 (193h5m33.609395727s) [noise=true , address=45.91.116.4:4939 , piece_id=KHSJLX5X3WLQTHB3ENOBLZNOHITMH5VOUYN5Q4RRIYWTO7LALUXA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:57.648938218 (193h5m33.688511348s) [noise=true , address=77.90.31.12:4589 , piece_id=FNIVFEHTPO2MG4YRUE7AILO2QIF4RCSBTIUQTK6HH5HVNHU5EEHA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:33:59.720595566 (193h5m35.760168686s) [noise=true , address=45.91.116.5:4298 , piece_id=LDWYCMSU6VF7ALA7LWMEFYMWSGPKDPOMRXZM2TDZ4WM4KLMM6JAA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.044112830 (193h5m36.08368596s) [noise=true , address=45.91.116.13:4155 , piece_id=R5FN2WFG5PJWC4VOWHWFRHX4SBOWFVRE7DLKPO2WI2CG3FCBLDSQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:00.626843171 (193h5m36.666416291s) [noise=true , address=145.79.227.7:4294 , piece_id=SBIZL4CTC34UUSXVJISN5ETW42CDCEMXLNP62PFEDKKZVOVT2HMA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.421592129 (193h5m37.461165259s) [noise=true , address=45.91.116.8:4045 , piece_id=5ZMTCXEFDB3Z5VRBEJ22NVCUI2MS6GDF54FE63CS7SN2QJ7OKD3A, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:01.502224723 (193h5m37.541797854s) [noise=true , address=45.91.116.13:4153 , piece_id=4OWS3TJKVYX2HROLNWLTBCOKO6GU5EU6W736RRP6PNX6GZJ22VUA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:02.637535080 (193h5m38.6771082s) [noise=true , address=45.91.116.14:4191 , piece_id=CYR7KZUXMZF3VHAQLSGJ6SYAVPZMX2G36CCTBFELVHSV72CQBBLA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.376187385 (193h5m48.415760505s) [noise=true , address=77.90.31.19:4842 , piece_id=BQTZBJ6XD4QSWS6QN4YZ4ODOPAAQQPXSGSAIKAETD52CYGCX3SJQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584873776 (193h5m48.624446896s) [noise=true , address=45.91.116.5:4299 , piece_id=HQYOQXXCASGVYTIQUSPKG6WZEOFTS656KPHZMHLFQSBXFDY6ZNUQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: <nil> Sep 14 14:34:12.584902090 (193h5m48.62447521s) [seg_pos={0 0}, upload=169699]: Segment reached optimal threshold of 13 pieces. Sep 14 14:34:12.585165033 (193h5m48.624738153s) [noise=true , address=45.91.116.11:4119 , piece_id=CC3QEJSESITL73WS3B2YZIIZTABJ42EXGPYL3F237JZ7ALQA7PTQ, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41268->45.91.116.11:4119: use of closed network connection Sep 14 14:34:12.585172387 (193h5m48.624745507s) [noise=true , address=45.91.116.10:4083 , piece_id=3LHXHRU6Q25WBJUYNUCXTQF4GZZBE3S77W2AYGXT5C7KJ4CIYHYA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36748->45.91.116.10:4083: use of closed network connection Sep 14 14:34:12.585199629 (193h5m48.624772749s) [noise=true , address=45.91.116.6:4973 , piece_id=56AF2LDGNAT55VGJ2JGJWYAPTRS7WOQM7CUW6V5FE7IZ6XXID3PA, seg_pos={0 0}, upload=169699]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58414->45.91.116.6:4973: use of closed network connection Sep 14 14:34:12.585233333 (193h5m48.624806453s) [seg_pos={0 0}, upload=169699]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 14 14:34:12.585295764 (193h5m48.624868884s) [upload=169699]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 14 14:34:12.585299611 (193h5m48.624872731s) [upload=169699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 14 14:34:13.053269081 (193h5m49.092842211s) [upload=169699]: Done uploading segments... Sep 14 14:34:13.053291775 (193h5m49.092864905s) [upload=169699]: Upload finished. err: <nil> Duration: 8.969322964s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.611224443s FAIL (2025-09-14 14:39:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.611245722s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)