Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.035247464s FAIL (2025-09-09 03:41:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.035278534s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.075593861s FAIL (2025-09-09 03:40:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.075687922s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.024845257s FAIL (2025-09-09 03:39:30 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.024870324s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.143247677s FAIL (2025-09-09 03:38:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.143276101s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.198528101s FAIL (2025-09-09 03:38:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.198559903s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.383609988s FAIL (2025-09-09 03:37:20 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.383630157s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000636936s FAIL (2025-09-09 03:37:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000657767s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.112443155s FAIL (2025-09-08 17:40:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.112472722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.941342322s FAIL (2025-09-08 17:39:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.941373482s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.529845966s FAIL (2025-09-08 17:38:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.529871515s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.959436682s FAIL (2025-09-08 17:37:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.959458102s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.041614093s FAIL (2025-09-08 17:36:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.041644091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.970725754s FAIL (2025-09-08 17:36:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.970751502s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.870222579s FAIL (2025-09-08 17:34:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.8702436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.02029853s FAIL (2025-09-08 17:34:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.020324932s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.979301066s FAIL (2025-09-08 17:33:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.979345552s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.11649546s FAIL (2025-09-08 17:32:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.116522622s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.926859908s FAIL (2025-09-08 17:32:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.926880528s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.096370569s FAIL (2025-09-08 17:31:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.096415134s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.87541153s FAIL (2025-09-08 17:30:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 5.875440385s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.081877753s FAIL (2025-09-08 17:30:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.081903823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 10.18574368s FAIL (2025-09-08 17:29:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 10.185771872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.159204786s FAIL (2025-09-08 17:27:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.159229433s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.724475973s FAIL (2025-09-08 17:27:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.724505249s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 7.734674123s FAIL (2025-09-08 17:26:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 7.734703999s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 7.431579728s FAIL (2025-09-08 17:25:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 7.431605677s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.211374986s FAIL (2025-09-08 17:24:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.211402299s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.069685719s FAIL (2025-09-08 17:23:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.069711678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.250363741s FAIL (2025-09-08 17:22:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.250395843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 8.267891571s FAIL (2025-09-08 17:21:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 8.267922932s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.348462944s FAIL (2025-09-08 17:20:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.348513681s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.031232212s FAIL (2025-09-08 17:19:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.031262259s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.315248501s FAIL (2025-09-08 17:18:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.315270833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.732697368s FAIL (2025-09-08 17:17:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.73271966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.511212922s FAIL (2025-09-08 17:16:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.511240445s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.01821199s FAIL (2025-09-08 17:15:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.018239501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.19929941s FAIL (2025-09-08 17:14:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.199332113s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.376390881s FAIL (2025-09-08 17:14:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.37642078s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 8.963685343s FAIL (2025-09-08 17:13:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 8.963714807s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 11.490874635s FAIL (2025-09-08 17:12:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 11.490902898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.358651038s FAIL (2025-09-08 17:12:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.358678942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.245964636s FAIL (2025-09-08 17:11:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.245992629s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.495940255s FAIL (2025-09-08 17:09:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.495978028s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.452612455s FAIL (2025-09-08 17:09:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.452641771s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X....................................X......................] duration: 6.194379359s FAIL (2025-09-08 17:08:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.194418644s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X....................................X.....................] duration: 6.308108398s FAIL (2025-09-08 17:08:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.308134287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X....................................X....................] duration: 6.711662319s FAIL (2025-09-08 17:07:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.711689391s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X....................................X...................] duration: 6.661397464s FAIL (2025-09-08 17:06:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.661420818s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X....................................X..................] duration: 6.1831122s FAIL (2025-09-08 17:05:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 6.18313793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X....................................X.................] duration: 10.552175586s FAIL (2025-09-08 17:05:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:53:46.497368971 (51h25m22.536942111s) [upload=45174]: Starting upload Sep 8 16:53:46.497478711 (51h25m22.537051831s) [upload=45174]: Done starting upload Sep 8 16:53:46.497560658 (51h25m22.537133808s) [upload=45174]: Uploading segments... Sep 8 16:53:46.497885229 (51h25m22.537458349s) [upload=45174]: Got next segment. Inline: false Sep 8 16:53:46.497917070 (51h25m22.5374902s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:53:46.498380978 (51h25m22.537954108s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:53:46.749320313 (51h25m22.788893433s) [seg_pos={0 0}, upload=45174]: Begin upload segment... Sep 8 16:53:46.749363946 (51h25m22.788937066s) [seg_pos={0 0}, upload=45174]: Done begin upload segment. Sep 8 16:53:46.749386500 (51h25m22.78895962s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749425564 (51h25m22.788998714s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.749981669 (51h25m22.789554839s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750048005 (51h25m22.789621155s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750307703 (51h25m22.789880823s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750566277 (51h25m22.790139387s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750643264 (51h25m22.790216385s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.750914995 (51h25m22.790488115s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751015557 (51h25m22.790588677s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751210510 (51h25m22.79078363s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751326542 (51h25m22.790899662s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751534180 (51h25m22.7911073s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751568435 (51h25m22.791141545s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751825837 (51h25m22.791398957s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.751900941 (51h25m22.791474061s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:46.752145940 (51h25m22.79171906s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Uploading piece... Sep 8 16:53:48.570928433 (51h25m24.610501563s) [upload=45174]: Next returned nil segment Sep 8 16:53:48.570948942 (51h25m24.610522072s) [upload=45174]: Waiting for error group managing segments... Sep 8 16:53:48.861067620 (51h25m24.90064074s) [noise=true , address=45.91.116.7:4011 , piece_id=CU6XXEK5ACDSOEQLUWCOEZ5ENFBZJMBKZZQGV67CDIHUJJRNSEEQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.881596814 (51h25m24.921169944s) [noise=true , address=45.91.116.9:4262 , piece_id=XVWUILXNJERHL7NZR2XKCG4EEJ2FP2RQDQ6V7CW3FUP3RIFGWB4Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.898201517 (51h25m24.937774637s) [noise=true , address=77.90.31.18:4807 , piece_id=WGGHRAU6VOPTRLANKFYSFAXFVC5WLFNV7X7VKQNWLOSYCKLU5LWA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.924075278 (51h25m24.963648398s) [noise=true , address=45.91.116.4:4939 , piece_id=TJKXRQOE3JHS2EN2N5LFSTIN66K5YBV6GHLHOWSVWUVXWKOCTSFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.925437615 (51h25m24.965010735s) [noise=true , address=77.90.31.13:4626 , piece_id=AVV3ZRYRJPBVEVVMQGPG4C42DTDOID2TUR64AG5XZXJT7T5MOYSQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.928921902 (51h25m24.968495022s) [noise=true , address=45.91.116.13:4155 , piece_id=KBUQUHX2QD74IP3QNYWRTBOXX6DRPA2XM7EKMTOHEVU7EQ5HB43A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.930896241 (51h25m24.970469361s) [noise=true , address=145.79.227.15:4873 , piece_id=GASAFOOEA3AQTDO6ZQC3OAOCXW46RURVXW2RD4HLKOCHGI6CZZJQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.943582225 (51h25m24.983155345s) [noise=true , address=45.91.116.6:4975 , piece_id=G7OOLDQEOARVDWG6KH4PNOZEDUDOYVWZXNJFCAZKIOZXSB24U3PA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.951253144 (51h25m24.990826264s) [noise=true , address=77.90.31.15:4699 , piece_id=RRF4O4W2B3C3GI65QM4LGYD2JET4JU22LAAVAUGMZ35VX2XRM3QA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.956196503 (51h25m24.995769623s) [noise=true , address=77.90.31.21:4879 , piece_id=3AFVC3W4IRYDO344MGOAHHE5WJEPBCZ67THOC6GMOI6RQX25ISBQ, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:48.985547198 (51h25m25.025120318s) [noise=true , address=77.90.31.16:4734 , piece_id=P5M5QWR4YVFZR6SH3BHNXG4O243BFDIA3JIB4DWG4NIYFRW5HXGA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.019429784 (51h25m25.059002904s) [noise=true , address=45.91.116.12:4225 , piece_id=SUK6ZBUCC5UAS36LIHJHIUQPVH3JFUT4JRHSNF3O6P2J6MHBZHZA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021285425 (51h25m25.060858545s) [noise=true , address=145.79.227.13:4804 , piece_id=SHGSPTOFYJKXICJDHYQHGJ62LPH6SPWGEOVVBBTHIXIS25KVMR3Q, seg_pos={0 0}, upload=45174]: Done uploading piece. err: <nil> Sep 8 16:53:49.021301856 (51h25m25.060874976s) [seg_pos={0 0}, upload=45174]: Segment reached optimal threshold of 13 pieces. Sep 8 16:53:49.021530242 (51h25m25.061103402s) [noise=true , address=45.91.116.11:4119 , piece_id=XXHON6P5H3FMKGVDTOTFP5NMMPUE5IZNNHA72AIA2PJCKTTHUUJA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45026->45.91.116.11:4119: use of closed network connection Sep 8 16:53:49.021552746 (51h25m25.061125866s) [noise=true , address=145.79.227.5:4221 , piece_id=BYPECHQFFEQ4RWYDVEFFXTHJVND2I5ZI732VYM5JK5GCTS6M4V5A, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Sep 8 16:53:49.021648219 (51h25m25.061221339s) [noise=true , address=45.91.116.14:4190 , piece_id=OR5CA7QROI7TNXE5J6J5DH6V55DII5DNBL5HGJAARUXXPQADEUFA, seg_pos={0 0}, upload=45174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: read tcp 188.166.225.170:45756->45.91.116.14:4190: use of closed network connection; context canceled Sep 8 16:53:49.021661273 (51h25m25.061234393s) [seg_pos={0 0}, upload=45174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:53:49.021731919 (51h25m25.061305039s) [upload=45174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:53:49.021735846 (51h25m25.061308966s) [upload=45174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:53:49.335802388 (51h25m25.375375508s) [upload=45174]: Done uploading segments... Sep 8 16:53:49.335821696 (51h25m25.375394816s) [upload=45174]: Upload finished. err: <nil> Duration: 10.552219861s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)