Orbiter History

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 647.825518ms target: https://us1.storj.io/login FAIL (2025-07-16 17:53:16 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 647.851007ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X................X..........X..........................] duration: 18.745564178s FAIL (2025-07-16 17:52:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 18.745592302s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 638.632589ms target: https://us1.storj.io/login FAIL (2025-07-16 17:52:49 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 638.654491ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X................X..........X.........................] duration: 5.989087203s FAIL (2025-07-16 17:52:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.989115095s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 640.576489ms target: https://us1.storj.io/login FAIL (2025-07-16 17:52:09 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 640.605294ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X................X..........X........................] duration: 5.986909354s FAIL (2025-07-16 17:51:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.986941114s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 642.700302ms target: https://us1.storj.io/login FAIL (2025-07-16 17:51:30 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 642.74058ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X................X..........X.......................] duration: 5.750762764s FAIL (2025-07-16 17:50:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.750790418s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 642.627674ms target: https://us1.storj.io/login FAIL (2025-07-16 17:50:19 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 642.654063ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X................X..........X......................] duration: 5.841590268s FAIL (2025-07-16 17:49:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.841614104s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 836.250818ms target: https://us1.storj.io/login FAIL (2025-07-16 17:49:35 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 836.276487ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X................X..........X.....................] duration: 5.740473787s FAIL (2025-07-16 17:49:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.740501913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X................X..........X....................] duration: 5.800266383s FAIL (2025-07-16 17:49:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.800296641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X................X..........X...................] duration: 5.706600817s FAIL (2025-07-16 17:48:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.706628239s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 649.517867ms target: https://us1.storj.io/login FAIL (2025-07-16 17:48:27 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 649.544288ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X................X..........X..................] duration: 5.7609585s FAIL (2025-07-16 17:47:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.760986715s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 723.744096ms target: https://us1.storj.io/login FAIL (2025-07-16 17:47:51 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 723.770866ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X................X..........X.................] duration: 5.79367534s FAIL (2025-07-16 17:47:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.793700139s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 645.612033ms target: https://us1.storj.io/login FAIL (2025-07-16 17:46:51 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 645.640738ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X................X..........X................] duration: 5.831085351s FAIL (2025-07-16 17:46:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.831109597s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X................X..........X...............] duration: 5.763390468s FAIL (2025-07-16 17:45:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.7634226s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 649.703583ms target: https://us1.storj.io/login FAIL (2025-07-16 17:45:42 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 649.725705ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X................X..........X..............] duration: 5.69936851s FAIL (2025-07-16 17:45:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.699397857s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 646.319887ms target: https://us1.storj.io/login FAIL (2025-07-16 17:44:36 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 646.340415ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X................X..........X.............] duration: 5.873354688s FAIL (2025-07-16 17:44:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.87338258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X................X..........X............] duration: 5.775137963s FAIL (2025-07-16 17:43:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.775155777s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 736.622637ms target: https://us1.storj.io/login FAIL (2025-07-16 17:43:33 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 736.642335ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 736.99054ms target: https://us1.storj.io/login FAIL (2025-07-16 17:43:01 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 737.010158ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X................X..........X...........] duration: 5.856355373s FAIL (2025-07-16 17:42:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.856381773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X................X..........X..........] duration: 5.855464776s FAIL (2025-07-16 17:42:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.855490534s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 779.18788ms target: https://us1.storj.io/login FAIL (2025-07-16 17:42:10 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 779.223887ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X................X..........X.........] duration: 5.771134845s FAIL (2025-07-16 17:41:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.771153971s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 641.251932ms target: https://us1.storj.io/login FAIL (2025-07-16 17:41:37 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 641.272791ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X................X..........X........] duration: 5.804508719s FAIL (2025-07-16 17:41:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.804538538s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 657.801997ms target: https://us1.storj.io/login FAIL (2025-07-16 17:40:46 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 657.821694ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X................X..........X.......] duration: 5.768222673s FAIL (2025-07-16 17:40:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.768255124s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 638.379586ms target: https://us1.storj.io/login FAIL (2025-07-16 17:39:35 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 638.410525ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X................X..........X......] duration: 5.805490589s FAIL (2025-07-16 17:39:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.805525015s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 871.713982ms target: https://us1.storj.io/login FAIL (2025-07-16 17:39:06 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 871.740864ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X................X..........X.....] duration: 5.805481342s FAIL (2025-07-16 17:38:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.805503834s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 639.763754ms target: https://us1.storj.io/login FAIL (2025-07-16 17:38:17 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 639.794974ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X................X..........X....] duration: 5.789931438s FAIL (2025-07-16 17:37:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.789954803s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 756.693138ms target: https://us1.storj.io/login FAIL (2025-07-16 17:37:17 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 756.726743ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X................X..........X...] duration: 5.865131361s FAIL (2025-07-16 17:36:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.865174613s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 815.503997ms target: https://us1.storj.io/login FAIL (2025-07-16 17:36:06 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 815.545918ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X................X..........X..] duration: 6.201510712s FAIL (2025-07-16 17:36:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 6.201534657s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X................X..........X.] duration: 6.190098982s FAIL (2025-07-16 17:35:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 6.190135602s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X................X..........X] duration: 30.230590928s FAIL (2025-07-16 17:34:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== tem_SegmentCommit Jul 16 17:33:17.710374760 (560h22m52.481016095s) [upload=448118]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 17:33:18.176441596 (560h22m52.947082961s) [upload=448118]: Done uploading segments... Jul 16 17:33:18.176464019 (560h22m52.947105364s) [upload=448118]: Upload finished. err: <nil> Jul 16 17:34:24.404990334 (560h23m59.175631669s) [upload=448131]: Starting upload Jul 16 17:34:24.405048284 (560h23m59.17568962s) [upload=448131]: Done starting upload Jul 16 17:34:24.405137725 (560h23m59.17577906s) [upload=448131]: Uploading segments... Jul 16 17:34:24.405225243 (560h23m59.175866578s) [upload=448131]: Got next segment. Inline: false Jul 16 17:34:24.405251323 (560h23m59.175892658s) [upload=448131]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 17:34:24.405255751 (560h23m59.175897086s) [upload=448131]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 17:34:24.746894352 (560h23m59.517535687s) [seg_pos={0 0}, upload=448131]: Begin upload segment... Jul 16 17:34:24.746940491 (560h23m59.517581816s) [seg_pos={0 0}, upload=448131]: Done begin upload segment. Jul 16 17:34:24.746967572 (560h23m59.517608908s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.746992390 (560h23m59.517633735s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747280310 (560h23m59.517921646s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747583881 (560h23m59.518225206s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.747873596 (560h23m59.518514931s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748056816 (560h23m59.518698151s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748429479 (560h23m59.519070814s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748770041 (560h23m59.519411376s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.748866055 (560h23m59.5195074s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749159557 (560h23m59.519800882s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749414144 (560h23m59.520055479s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749526007 (560h23m59.520167343s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749651809 (560h23m59.520293144s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.749903189 (560h23m59.520544524s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.750047255 (560h23m59.52068859s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:24.752454691 (560h23m59.523096026s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Uploading piece... Jul 16 17:34:26.444711640 (560h24m1.215352975s) [upload=448131]: Next returned nil segment Jul 16 17:34:26.444757939 (560h24m1.215399274s) [upload=448131]: Waiting for error group managing segments... Jul 16 17:34:26.792535617 (560h24m1.563176952s) [noise=true , address=104.36.82.8:4972 , piece_id=32G7DXHJC4BC4AVFDHQ2AGIUAKVAU3MRKRGHV43OAHBFUSVFOGEQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.794159945 (560h24m1.56480128s) [noise=true , address=104.36.82.13:4185 , piece_id=7DDQHMP63IZ7VKTP6AWSURELFA7TZOPRM374IEREBICZ52SQK7IQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.810601202 (560h24m1.581242537s) [noise=true , address=104.36.82.3:4767 , piece_id=LPM2KMZNPVVJ2OBAB2AJD3SXI3QQ54YQHO7FZTXY64OISUMF3RLA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.812727070 (560h24m1.583368405s) [noise=true , address=104.36.82.6:4873 , piece_id=PA5Q5EYREVNVS4BOKJQRHECNNRXAHOYAXMBNQNXQKT7U42FMAJXQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.881085142 (560h24m1.651726477s) [noise=true , address=104.36.82.16:4294 , piece_id=RWB2PZDI2ID6SPD2HUSIRZYR4VSYNVQ2CCARGSH5WY5BXWYONUKQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.895742627 (560h24m1.666383963s) [noise=true , address=104.36.82.11:4115 , piece_id=OX4MPFH372FGFSYOED63JXZDTJNCX3CZLGUEAI7EMDBUP2DCZVZA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.953858663 (560h24m1.724499998s) [noise=true , address=104.36.82.5:4837 , piece_id=A3U2EB5QKVWKGPWDDBN7OEFAW2BISATXR7JSYNJMOVIABH5KW6SA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.958737709 (560h24m1.729379034s) [noise=true , address=104.36.82.10:4077 , piece_id=CPNT7XOOQT4GMPYUL4KS7HUNLF4NB42HVT7SPERN3QPZUWYGSOSA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.962052380 (560h24m1.732693715s) [noise=true , address=104.36.82.9:4007 , piece_id=ZU7MVTDTKSLFQNDFZBIY42YF3KUVDWSZKB6VDKZKM6XOL3JMLZRA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.966448942 (560h24m1.737090267s) [noise=true , address=104.36.82.2:4732 , piece_id=GEVJ235LSMKPK2I46CZZYDJWUJTYOOIK7QSLFDPMP37ABOMDJJDA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:26.996082532 (560h24m1.766723877s) [noise=true , address=104.36.82.11:4114 , piece_id=J52V6D3YSK3TVZDYUFEF2VAEKZ35QSIN7GWVA7LRW6L2T2WFEJTA, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:27.067684761 (560h24m1.838326146s) [noise=true , address=45.91.116.5:4299 , piece_id=MBGWZOZSY63LLV4DK7VHEIQVFJ4CS6FF3B7T2T6CQSPK7HDXY76A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: <nil> Jul 16 17:34:53.765713115 (560h24m28.536354451s) [noise=true , address=77.90.31.12:4592 , piece_id=BE3EGFPVK7OE4FOFKWRXEGGT3MHM7L3F47XMDB5XGHS5SEPPWCIQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42636->77.90.31.12:4592: use of closed network connection Jul 16 17:34:53.765770917 (560h24m28.536412252s) [noise=true , address=77.90.31.15:4699 , piece_id=M7GQWLBVZHPRB2VVKQSAVBB7A2LGVB46XRLBPLF27NJIXL4Z5QMQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35344->77.90.31.15:4699: use of closed network connection Jul 16 17:34:53.765782188 (560h24m28.536423523s) [noise=true , address=77.90.31.19:4842 , piece_id=EMWJAA6GZKVG2CZ2CNLHUBEJYBH752ER6RXBVTEAJL44ZMH4D7AQ, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:59270->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 16 17:34:53.765860638 (560h24m28.536501973s) [noise=true , address=77.90.31.11:4553 , piece_id=YI3MPUWTW7BYGYR3SZUYFIM2USBY367YV5CNZYM47E2Z7V6HDT3A, seg_pos={0 0}, upload=448131]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57362->77.90.31.11:4553: use of closed network connection Jul 16 17:34:53.765891447 (560h24m28.536532782s) [seg_pos={0 0}, upload=448131]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 17:34:53.994663536 (560h24m28.765304941s) [upload=448131]: Done uploading segments... Jul 16 17:34:53.994695617 (560h24m28.765336952s) [upload=448131]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 30.230612889s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 892.333359ms target: https://us1.storj.io/login FAIL (2025-07-16 17:34:53 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 892.351343ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 642.293819ms target: https://us1.storj.io/login FAIL (2025-07-16 17:33:43 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused Duration: 642.321832ms core satellite us1

Orbiter Check History

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)