Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................X.........X.................X.........] duration: 1.861680471s FAIL (2025-07-26 21:41:06 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.861732881s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 1m0.232330164s FAIL (2025-07-26 21:40:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== 737]: Uploading piece... Jul 26 21:39:57.359356920 (58h4m7.08191995s) [noise=true , address=104.36.82.56:4254 , piece_id=YESLEBWMOXSUCTMVZKDRPY7R5HQJFPDY3MFXNNL5SC7GVQX7Z3MQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.359716187 (58h4m7.082279228s) [noise=true , address=104.36.82.52:4088 , piece_id=EC35PJ2F6TMUCXPYEGWDGMOXYDZVQTMXCPEPCFLI65FZZJ2DLGZA, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.359926130 (58h4m7.08248917s) [noise=true , address=77.90.31.10:4531 , piece_id=5G6EQ66OMVAEU44WDC2HRM4QCYCX2ESSSTWEPPKPHRAK6SGBDQKQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.360166759 (58h4m7.082729789s) [noise=true , address=104.36.82.47:4532 , piece_id=BNYQH5X44AWGUVN7D6LTEO3CWK26723QOWOWMHKOVARPBG7NZFVA, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.360445313 (58h4m7.083008353s) [noise=true , address=104.36.82.46:4483 , piece_id=JWGCN3Y4WGH4LBK2FVZGFWUE3JA5P2CT2MRNJSF2ZAAMDZATBSXA, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.360470752 (58h4m7.083033792s) [noise=true , address=104.36.82.49:4002 , piece_id=RYXFYPMYAEWIWV22OBHFTHYSYJC3MOBI2MIMPSTJH7TESLQRD6QA, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.360859125 (58h4m7.083422165s) [noise=true , address=79.127.205.246:4282 , piece_id=7OFKJR2BNRPBXV52T65QM2JFWJBTLD2523INMY6GHFSC2KUVPT6Q, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.361079356 (58h4m7.083642386s) [noise=true , address=79.127.201.215:4360 , piece_id=6PJDK35F43CTAAY44XU34QVHI3VHMI2EVAGUP5J7VUOH3EBPHNLQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.361212360 (58h4m7.0837754s) [noise=true , address=104.36.82.54:4167 , piece_id=MMMJAY7K7J5UQZVQI2IKYXSSD2TX2P5GRQP4W3R3Q3WPEWMRFTQQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.361450796 (58h4m7.084013836s) [noise=true , address=104.36.82.21:4482 , piece_id=LZF3WIKQYVAGWNJDR2NRI2JW2J4KTJDOQ5V3JSCUKQIFS7DOPVEQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.361641441 (58h4m7.084204481s) [noise=true , address=79.127.205.245:4223 , piece_id=XPI2362SSDZFQX77Z42KEPM4D5REEOCYKKFTCRFSHIUGLN4W4VQA, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.361908232 (58h4m7.084471272s) [noise=true , address=77.90.31.12:4608 , piece_id=P4EYX6ADFSAMISEYWCHSMH27SRQABGNICDHAAXGJQI6Y4PONLE4A, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.362084499 (58h4m7.084647539s) [noise=true , address=104.36.82.6:4898 , piece_id=C3BTMXA2XXGMAMWGUUHVLKHKMT2XIBLYMNFK4R4JXOKKUNBRNMDQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:57.362302947 (58h4m7.084865987s) [noise=true , address=104.36.82.12:4168 , piece_id=UHBFZCEJW5QV5AJKLCMYV3MIB2OZCW5EKLUPCDCH6ONUTILC7ZYQ, seg_pos={0 0}, upload=55737]: Uploading piece... Jul 26 21:39:58.464880053 (58h4m8.187443093s) [upload=55737]: Next returned nil segment Jul 26 21:39:58.464896926 (58h4m8.187459966s) [upload=55737]: Waiting for error group managing segments... Jul 26 21:39:58.788907026 (58h4m8.511470066s) [noise=true , address=79.127.205.245:4223 , piece_id=XPI2362SSDZFQX77Z42KEPM4D5REEOCYKKFTCRFSHIUGLN4W4VQA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:39:59.005645121 (58h4m8.728208161s) [noise=true , address=79.127.205.246:4282 , piece_id=7OFKJR2BNRPBXV52T65QM2JFWJBTLD2523INMY6GHFSC2KUVPT6Q, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:40:00.823730002 (58h4m10.546293043s) [noise=true , address=79.127.219.38:4004 , piece_id=7C4BW4KITP2WSUJQ54KQ4Q5ON2SCDE3GF7BIIKX7W7RFN5246LMQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:40:02.058589587 (58h4m11.781152637s) [noise=true , address=77.90.31.12:4608 , piece_id=P4EYX6ADFSAMISEYWCHSMH27SRQABGNICDHAAXGJQI6Y4PONLE4A, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:40:02.670760586 (58h4m12.393323626s) [noise=true , address=77.90.31.10:4531 , piece_id=5G6EQ66OMVAEU44WDC2HRM4QCYCX2ESSSTWEPPKPHRAK6SGBDQKQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:40:02.684182716 (58h4m12.406745756s) [noise=true , address=79.127.201.215:4360 , piece_id=6PJDK35F43CTAAY44XU34QVHI3VHMI2EVAGUP5J7VUOH3EBPHNLQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:40:03.213377830 (58h4m12.93594087s) [noise=true , address=64.49.13.154:4928 , piece_id=DLBKVBRYK2U3WQG6RWYHEU4OGQBBPEXAIVV6E2IJQ55PVLGITPIA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: <nil> Jul 26 21:40:56.279213773 (58h5m6.001776813s) [noise=true , address=104.36.82.54:4167 , piece_id=MMMJAY7K7J5UQZVQI2IKYXSSD2TX2P5GRQP4W3R3Q3WPEWMRFTQQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ, address:104.36.82.54:4167): protocol: rpc: dial tcp 104.36.82.54:4167: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279258038 (58h5m6.001821078s) [noise=true , address=104.36.82.42:4814 , piece_id=FQVRE4RFIPDQWY3ATRYOI6VTQROD3LKOGPW74CA3PBAJRDABAA4Q, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:12rnmX5iuA5swKceTZVV14HLaUctRFimn3DPLjRogg4K3N7PWnU, address:104.36.82.42:4814): protocol: rpc: dial tcp 104.36.82.42:4814: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279289759 (58h5m6.001852799s) [noise=true , address=104.36.82.49:4002 , piece_id=RYXFYPMYAEWIWV22OBHFTHYSYJC3MOBI2MIMPSTJH7TESLQRD6QA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo, address:104.36.82.49:4002): protocol: rpc: dial tcp 104.36.82.49:4002: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279662502 (58h5m6.002225543s) [noise=true , address=104.36.82.55:4218 , piece_id=V5DGG7WGAQNQZNP3BUJTMHIZ3VPPKI2LATKODYI3QOCEB2WYPDTA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:104.36.82.55:4218): protocol: write tcp 188.166.225.170:45896->104.36.82.55:4218: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279693321 (58h5m6.002256361s) [noise=true , address=104.36.82.46:4483 , piece_id=JWGCN3Y4WGH4LBK2FVZGFWUE3JA5P2CT2MRNJSF2ZAAMDZATBSXA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:1igSs2ZGX3zWVsSdmm1pnwYd7JckcyBubTuNU1gJDxQsZ9nKaY, address:104.36.82.46:4483): protocol: rpc: dial tcp 104.36.82.46:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279723248 (58h5m6.002286289s) [noise=true , address=104.36.82.6:4898 , piece_id=C3BTMXA2XXGMAMWGUUHVLKHKMT2XIBLYMNFK4R4JXOKKUNBRNMDQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:1D4E49RAbRK64c95rK6tGHPHHo5xLsmZ53ZK9i2iRU8mz7SbZ8, address:104.36.82.6:4898): protocol: rpc: dial tcp 104.36.82.6:4898: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279751472 (58h5m6.002314513s) [noise=true , address=104.36.82.52:4088 , piece_id=EC35PJ2F6TMUCXPYEGWDGMOXYDZVQTMXCPEPCFLI65FZZJ2DLGZA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:1284h6cPNVXJYMkSekGQVw8b7p89tkhpmgYLxTTYUDu2J7Wadoj, address:104.36.82.52:4088): protocol: write tcp 188.166.225.170:44928->104.36.82.52:4088: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279779366 (58h5m6.002342406s) [noise=true , address=104.36.82.12:4168 , piece_id=UHBFZCEJW5QV5AJKLCMYV3MIB2OZCW5EKLUPCDCH6ONUTILC7ZYQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:12XJ5kTKY8pLq3uDJZfFVkTgS56Xx8pG2R5i2A9VRNDEckdKnTF, address:104.36.82.12:4168): protocol: rpc: dial tcp 104.36.82.12:4168: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279882353 (58h5m6.002445393s) [noise=true , address=104.36.82.21:4482 , piece_id=LZF3WIKQYVAGWNJDR2NRI2JW2J4KTJDOQ5V3JSCUKQIFS7DOPVEQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:12B2s428CMVK7mdLV84VgWFcdRQaVQxcA7YuqqeqEM8FAefEGt7, address:104.36.82.21:4482): protocol: write tcp 188.166.225.170:33746->104.36.82.21:4482: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279903414 (58h5m6.002466454s) [noise=true , address=104.36.82.56:4254 , piece_id=YESLEBWMOXSUCTMVZKDRPY7R5HQJFPDY3MFXNNL5SC7GVQX7Z3MQ, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:1k4x3HL3PNHwKJX83RyqvnsATHk3HmeMiC12A7raLN58wzvkUN, address:104.36.82.56:4254): protocol: write tcp 188.166.225.170:53458->104.36.82.56:4254: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279932740 (58h5m6.00249578s) [noise=true , address=104.36.82.47:4532 , piece_id=BNYQH5X44AWGUVN7D6LTEO3CWK26723QOWOWMHKOVARPBG7NZFVA, seg_pos={0 0}, upload=55737]: Done uploading piece. err: ecclient: upload failed (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL, address:104.36.82.47:4532): protocol: write tcp 188.166.225.170:37406->104.36.82.47:4532: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:40:56.279946145 (58h5m6.002509185s) [seg_pos={0 0}, upload=55737]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 26 21:40:56.509340325 (58h5m6.231903365s) [upload=55737]: Done uploading segments... Jul 26 21:40:56.509365433 (58h5m6.231928473s) [upload=55737]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.232346376s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X...................................XX........] duration: 11.853893365s FAIL (2025-07-26 21:40:55 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 11.853921018s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.........X.................X........] duration: 1.899744672s FAIL (2025-07-26 21:40:29 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.899787634s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................X....X......XX.] duration: 11.32591912s FAIL (2025-07-26 21:40:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== .605925663 (58h3m0.328488703s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:38:50.605944900 (58h3m0.32850794s) [seg_pos={0 0}, upload=55710]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:50.833093218 (58h3m0.555656258s) [upload=55710]: Done uploading segments... Jul 26 21:38:50.833138233 (58h3m0.555701293s) [upload=55710]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:56.916704322 (58h3m6.639267363s) [upload=55721]: Starting upload Jul 26 21:38:56.916756792 (58h3m6.639319832s) [upload=55721]: Done starting upload Jul 26 21:38:56.916934593 (58h3m6.639497633s) [upload=55721]: Uploading segments... Jul 26 21:38:56.916968998 (58h3m6.639532028s) [upload=55721]: Got next segment. Inline: false Jul 26 21:38:56.916992904 (58h3m6.639555934s) [upload=55721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:56.916997673 (58h3m6.639560713s) [upload=55721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:57.151780998 (58h3m6.874344038s) [seg_pos={0 0}, upload=55721]: Begin upload segment... Jul 26 21:38:57.151833869 (58h3m6.87439693s) [seg_pos={0 0}, upload=55721]: Done begin upload segment. Jul 26 21:38:57.151846042 (58h3m6.874409082s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.151855912 (58h3m6.874418982s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152377529 (58h3m6.874940569s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152837629 (58h3m6.87540067s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.153782999 (58h3m6.876346029s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154085317 (58h3m6.876648347s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154213682 (58h3m6.876776722s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154448712 (58h3m6.877011752s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154666529 (58h3m6.877229569s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154892101 (58h3m6.877455141s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155080802 (58h3m6.877643842s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155274713 (58h3m6.877837753s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155489684 (58h3m6.878052724s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155906972 (58h3m6.878470012s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156106574 (58h3m6.878669614s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156330883 (58h3m6.878893923s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:58.895021080 (58h3m8.61758415s) [upload=55721]: Next returned nil segment Jul 26 21:38:58.895042832 (58h3m8.617605902s) [upload=55721]: Waiting for error group managing segments... Jul 26 21:38:59.168496886 (58h3m8.891059926s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.177587149 (58h3m8.900150189s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.256356711 (58h3m8.978919751s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.263143385 (58h3m8.985706425s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.277648448 (58h3m9.000211488s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.279279730 (58h3m9.00184277s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.307593714 (58h3m9.030156754s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.314667739 (58h3m9.037230779s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.349724434 (58h3m9.072287464s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:04.104309440 (58h3m13.82687248s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:05.457684029 (58h3m15.18024707s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:26.274464249 (58h3m35.997027289s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274493555 (58h3m35.997056595s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:60166->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274509737 (58h3m35.997072777s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274528824 (58h3m35.997091864s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274548020 (58h3m35.99711106s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 188.166.225.170:56794->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274580142 (58h3m35.997143172s) [seg_pos={0 0}, upload=55721]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:39:26.511752419 (58h3m36.234315449s) [upload=55721]: Done uploading segments... Jul 26 21:39:26.511771585 (58h3m36.234334625s) [upload=55721]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 11.32594539s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X...................................XX.......] duration: 25.123937214s FAIL (2025-07-26 21:40:05 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 25.123965088s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 1m0.232705691s FAIL (2025-07-26 21:39:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== 351s) [noise=true , address=104.36.82.53:4146 , piece_id=RWCPGZCHTR4C5EZJEERKFY3PFAR5D5AZNYGUSN7R3GHHZ4FZFHVQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.917733752 (58h3m6.640296792s) [noise=true , address=104.36.82.50:4026 , piece_id=EIN62QNK4AZTHFKNVAUGQU4IC36LRY5BYS7EW2LJRLPKMOO4S5ZQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.917874641 (58h3m6.640437681s) [noise=true , address=77.90.31.12:4608 , piece_id=GJZSWUDS47R5QLD3T34ALR2EOJZYVW7VQZDP6FHINWOZQ5TJUMLA, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918017093 (58h3m6.640580194s) [noise=true , address=104.36.82.51:4060 , piece_id=Q5SIMYUIK6HYC6AK57R7JEB7YFGBCYE6J2MVQF6K7FUEFFIZ6SPQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918138146 (58h3m6.640701186s) [noise=true , address=104.36.82.49:4979 , piece_id=3OCKOABFHWHYZV2NS5KDE377NRJGE354J3CKDAXKDNSUOQS27N2A, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918396429 (58h3m6.64095947s) [noise=true , address=104.36.82.11:4127 , piece_id=MNOBBDTL7P5JRMLCB6YZRP626KSJJ6FQ7HLC6U2RBX7UMNSF7KOQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918455913 (58h3m6.641018953s) [noise=true , address=104.36.82.39:4760 , piece_id=HWODLETT2GUPNLAU3BGOGXIOKAMROCNUIGVHSUX2FPGO767JNOQQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918614517 (58h3m6.641177557s) [noise=true , address=79.127.205.235:4679 , piece_id=LXH2XDHJTDHQQLDWDDYWS3UNI77IY5JCGHSS4OD6ZUIZSHOREKKQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918676405 (58h3m6.641239435s) [noise=true , address=104.36.82.20:4457 , piece_id=RMPWVA5YDSCH36LLXTZQJUCJA725JULHUI2KF3CXGFSHRB3OTGHQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918884664 (58h3m6.641447704s) [noise=true , address=104.36.82.19:4407 , piece_id=5NTZ3RJRMIHQLZT5SGR5OIFI2UZNOQFGFMNV6ENH34NVUYE4DQQA, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.918895133 (58h3m6.641458174s) [noise=true , address=104.36.82.55:4216 , piece_id=STW2XMALV6ZVVV5PY4N74TT5OU23YH5TZP5PS6GGPRX7P6VXTGHQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.919118471 (58h3m6.641681501s) [noise=true , address=104.36.82.16:4316 , piece_id=MLTMKQBDWOZNV72646CGPEG34GLJU2EHGSZ57PMBNGMLIE3NQ5KQ, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.919122769 (58h3m6.641685799s) [noise=true , address=104.36.82.52:4090 , piece_id=HSZKIQ52RY3SQK365RCNSPVYAKJT2P6UFBCYIKULXTYTBQPGGADA, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:56.919350625 (58h3m6.641913665s) [noise=true , address=104.36.82.56:4239 , piece_id=JEPYQQ7AAU2BGD2D7CDIX66A5TW6ZUHYXSMULK7XHWF64PDCNXZA, seg_pos={0 0}, upload=55720]: Uploading piece... Jul 26 21:38:58.059355161 (58h3m7.781918201s) [upload=55720]: Next returned nil segment Jul 26 21:38:58.059373266 (58h3m7.781936306s) [upload=55720]: Waiting for error group managing segments... Jul 26 21:38:58.362317475 (58h3m8.084880515s) [noise=true , address=79.127.205.235:4679 , piece_id=LXH2XDHJTDHQQLDWDDYWS3UNI77IY5JCGHSS4OD6ZUIZSHOREKKQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: <nil> Jul 26 21:38:58.385169899 (58h3m8.10773294s) [noise=true , address=79.127.205.234:4606 , piece_id=VRQNB6Z4UN4ZWHOVJ6LAYXU63NWJWGRSJARDYV7H32WRE4632YNA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: <nil> Jul 26 21:39:00.857562001 (58h3m10.580125041s) [noise=true , address=79.127.201.215:4344 , piece_id=72J42KMDS2NFGX27ZRSKMBIELE3VXN3U2K6CYNC22SS6EG7R24TA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: <nil> Jul 26 21:39:03.655742248 (58h3m13.378305288s) [noise=true , address=77.90.31.12:4608 , piece_id=GJZSWUDS47R5QLD3T34ALR2EOJZYVW7VQZDP6FHINWOZQ5TJUMLA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: <nil> Jul 26 21:39:26.957156158 (58h3m36.679719198s) [noise=true , address=104.36.82.50:4026 , piece_id=EIN62QNK4AZTHFKNVAUGQU4IC36LRY5BYS7EW2LJRLPKMOO4S5ZQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: <nil> Jul 26 21:39:34.983435033 (58h3m44.705998073s) [noise=true , address=104.36.82.20:4457 , piece_id=RMPWVA5YDSCH36LLXTZQJUCJA725JULHUI2KF3CXGFSHRB3OTGHQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: <nil> Jul 26 21:39:55.817990657 (58h4m5.540553697s) [noise=true , address=104.36.82.56:4239 , piece_id=JEPYQQ7AAU2BGD2D7CDIX66A5TW6ZUHYXSMULK7XHWF64PDCNXZA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:126Feu3gWgog12YavTHZkkUKK8wq6RynV8zsWd3WUAUYjvQwBnc, address:104.36.82.56:4239): protocol: rpc: dial tcp 104.36.82.56:4239: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818034810 (58h4m5.54059785s) [noise=true , address=104.36.82.36:4622 , piece_id=K3CJEGV7BAGUAW7WSOFTYNB37NRDOI2YFR4BPPDD6MNQXMS67TXA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:1xEK5XL3sYcip7PsQcZBQYihgnEZykdMmsKkCr2TUmbTFYncfs, address:104.36.82.36:4622): protocol: rpc: dial tcp 104.36.82.36:4622: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818068145 (58h4m5.540631185s) [noise=true , address=104.36.82.54:4185 , piece_id=D6QDRYQ2TT2FCJ66UNQUGDMAFC52IQDVKKAAQAVLSW33VO6CMTZA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:1cLdyyVLrp1yBmgsfBrfYc6N2RFbBichYoBbo2F7EZfaBXtkfk, address:104.36.82.54:4185): protocol: rpc: dial tcp 104.36.82.54:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818097681 (58h4m5.540660721s) [noise=true , address=104.36.82.39:4760 , piece_id=HWODLETT2GUPNLAU3BGOGXIOKAMROCNUIGVHSUX2FPGO767JNOQQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:1jJcppBK9zJzGedgY9KEjEA8twqtkdZcjhcnXJoN5AWi5QzR4E, address:104.36.82.39:4760): protocol: rpc: dial tcp 104.36.82.39:4760: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818130304 (58h4m5.540693344s) [noise=true , address=104.36.82.53:4146 , piece_id=RWCPGZCHTR4C5EZJEERKFY3PFAR5D5AZNYGUSN7R3GHHZ4FZFHVQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:1ecwZFchyb5HmyKwRFfLDaVGck3xDWmgrVNYadHoTLWY5Z2Qs1, address:104.36.82.53:4146): protocol: rpc: dial tcp 104.36.82.53:4146: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818160020 (58h4m5.54072305s) [noise=true , address=104.36.82.11:4127 , piece_id=MNOBBDTL7P5JRMLCB6YZRP626KSJJ6FQ7HLC6U2RBX7UMNSF7KOQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:12Pxos2xcjHwcYyQnXeRL3PL72G9pXaAnBs8XnkVFkxegDRPrqp, address:104.36.82.11:4127): protocol: rpc: dial tcp 104.36.82.11:4127: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818264240 (58h4m5.54082728s) [noise=true , address=104.36.82.16:4316 , piece_id=MLTMKQBDWOZNV72646CGPEG34GLJU2EHGSZ57PMBNGMLIE3NQ5KQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:12PPnXDzwXDVDKemw483GXuzQrPN48JJrjeeD7az2NPNawHNaXf, address:104.36.82.16:4316): protocol: rpc: dial tcp 104.36.82.16:4316: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818278006 (58h4m5.540841046s) [noise=true , address=104.36.82.51:4060 , piece_id=Q5SIMYUIK6HYC6AK57R7JEB7YFGBCYE6J2MVQF6K7FUEFFIZ6SPQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:19VJKnTAges9YKYdUVoz3tb3sFU1bawxpaJ2rsBd2wmLo1zK4X, address:104.36.82.51:4060): protocol: rpc: dial tcp 104.36.82.51:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818318133 (58h4m5.540881173s) [noise=true , address=104.36.82.55:4216 , piece_id=STW2XMALV6ZVVV5PY4N74TT5OU23YH5TZP5PS6GGPRX7P6VXTGHQ, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:12vYNo8tVKDfSphJzsryTLhezUDVw3DA1NaK7ZXcXYcEw4pf1Pt, address:104.36.82.55:4216): protocol: rpc: dial tcp 104.36.82.55:4216: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818356346 (58h4m5.540919386s) [noise=true , address=104.36.82.52:4090 , piece_id=HSZKIQ52RY3SQK365RCNSPVYAKJT2P6UFBCYIKULXTYTBQPGGADA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:12UUnoD9p5FC7rq9fMSzu1phkFpsnPsu4UYxC59MhpMjJ6wUC6F, address:104.36.82.52:4090): protocol: rpc: dial tcp 104.36.82.52:4090: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818363079 (58h4m5.540926119s) [noise=true , address=104.36.82.49:4979 , piece_id=3OCKOABFHWHYZV2NS5KDE377NRJGE354J3CKDAXKDNSUOQS27N2A, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:12B518TvbbRhj6LiXxV4GwUgAoUYmWMWeTpyYerKnKuZYBjXoma, address:104.36.82.49:4979): protocol: write tcp 188.166.225.170:35170->104.36.82.49:4979: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818398036 (58h4m5.540961076s) [noise=true , address=104.36.82.19:4407 , piece_id=5NTZ3RJRMIHQLZT5SGR5OIFI2UZNOQFGFMNV6ENH34NVUYE4DQQA, seg_pos={0 0}, upload=55720]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:104.36.82.19:4407): protocol: rpc: dial tcp 104.36.82.19:4407: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:55.818431770 (58h4m5.540994801s) [seg_pos={0 0}, upload=55720]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:39:56.048128049 (58h4m5.770691089s) [upload=55720]: Done uploading segments... Jul 26 21:39:56.048154219 (58h4m5.770717259s) [upload=55720]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.232728044s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................X.........X.................X.......] duration: 1.854436143s FAIL (2025-07-26 21:39:51 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.85446564s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................X....X......XX] duration: 30.2391277s FAIL (2025-07-26 21:39:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== .605925663 (58h3m0.328488703s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:38:50.605944900 (58h3m0.32850794s) [seg_pos={0 0}, upload=55710]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:50.833093218 (58h3m0.555656258s) [upload=55710]: Done uploading segments... Jul 26 21:38:50.833138233 (58h3m0.555701293s) [upload=55710]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:56.916704322 (58h3m6.639267363s) [upload=55721]: Starting upload Jul 26 21:38:56.916756792 (58h3m6.639319832s) [upload=55721]: Done starting upload Jul 26 21:38:56.916934593 (58h3m6.639497633s) [upload=55721]: Uploading segments... Jul 26 21:38:56.916968998 (58h3m6.639532028s) [upload=55721]: Got next segment. Inline: false Jul 26 21:38:56.916992904 (58h3m6.639555934s) [upload=55721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:56.916997673 (58h3m6.639560713s) [upload=55721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:57.151780998 (58h3m6.874344038s) [seg_pos={0 0}, upload=55721]: Begin upload segment... Jul 26 21:38:57.151833869 (58h3m6.87439693s) [seg_pos={0 0}, upload=55721]: Done begin upload segment. Jul 26 21:38:57.151846042 (58h3m6.874409082s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.151855912 (58h3m6.874418982s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152377529 (58h3m6.874940569s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152837629 (58h3m6.87540067s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.153782999 (58h3m6.876346029s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154085317 (58h3m6.876648347s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154213682 (58h3m6.876776722s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154448712 (58h3m6.877011752s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154666529 (58h3m6.877229569s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154892101 (58h3m6.877455141s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155080802 (58h3m6.877643842s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155274713 (58h3m6.877837753s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155489684 (58h3m6.878052724s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155906972 (58h3m6.878470012s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156106574 (58h3m6.878669614s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156330883 (58h3m6.878893923s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:58.895021080 (58h3m8.61758415s) [upload=55721]: Next returned nil segment Jul 26 21:38:58.895042832 (58h3m8.617605902s) [upload=55721]: Waiting for error group managing segments... Jul 26 21:38:59.168496886 (58h3m8.891059926s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.177587149 (58h3m8.900150189s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.256356711 (58h3m8.978919751s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.263143385 (58h3m8.985706425s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.277648448 (58h3m9.000211488s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.279279730 (58h3m9.00184277s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.307593714 (58h3m9.030156754s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.314667739 (58h3m9.037230779s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.349724434 (58h3m9.072287464s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:04.104309440 (58h3m13.82687248s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:05.457684029 (58h3m15.18024707s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:26.274464249 (58h3m35.997027289s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274493555 (58h3m35.997056595s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:60166->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274509737 (58h3m35.997072777s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274528824 (58h3m35.997091864s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274548020 (58h3m35.99711106s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 188.166.225.170:56794->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274580142 (58h3m35.997143172s) [seg_pos={0 0}, upload=55721]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:39:26.511752419 (58h3m36.234315449s) [upload=55721]: Done uploading segments... Jul 26 21:39:26.511771585 (58h3m36.234334625s) [upload=55721]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.239147266s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................X.........X.................X......] duration: 1.871803994s FAIL (2025-07-26 21:39:15 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.871838841s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 1m0.231206833s FAIL (2025-07-26 21:38:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== 6 21:37:52.433840671 (58h2m2.156403701s) [noise=true , address=104.36.82.56:4240 , piece_id=HZJ3DMH6WLLTIVOSRMHBVU6O3B7GYFOLSEEZAMU4YBPQ4OE67X4A, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.433982171 (58h2m2.156545211s) [noise=true , address=104.36.82.54:4170 , piece_id=FFJTYTCJX32QIVS2LKYZHIWNNM35FJZU6F5S4WPZGGXVQ37TF42A, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.434091110 (58h2m2.15665415s) [noise=true , address=104.36.82.52:4102 , piece_id=JBZ4R5KAX2O5VKSM2P4OW6XMTS77IK6DNWOTGOU4EK7XSLOX66TQ, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.434338102 (58h2m2.156901142s) [noise=true , address=45.91.116.14:4222 , piece_id=UXEKJAXAOEILZT6ENTDKQMJUAVS2BQ2HTA6B5TDVAGUUCOCIOD3A, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.434570858 (58h2m2.157133898s) [noise=true , address=79.127.205.234:4612 , piece_id=YGPM7PFBTHUDM5EN5OZBXWBDAMBIQM6SG5EV2UIGVGSTPZLPWW3A, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.434781490 (58h2m2.15734453s) [noise=true , address=104.36.82.3:4780 , piece_id=63JWNPKTNIFCEM4J47S4YNAQGNPJZNMFJ654RPOWN43Q6LKSNFSA, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.435604515 (58h2m2.158167555s) [noise=true , address=104.36.82.53:4130 , piece_id=XOVNSIS3BQS5JQHMQPKP6MDV6M4XOLISJRYXESJPUG32FVBUSI3Q, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.435861608 (58h2m2.158424648s) [noise=true , address=104.36.82.46:4492 , piece_id=CBQI2NY5XCSJ3EONPVV3VLBATEFV2E3MXM4QLICCBVRA5QH4UHWQ, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.436103170 (58h2m2.15866621s) [noise=true , address=45.91.116.10:4111 , piece_id=7SIGL2STGEMEJFACJXWADLA5JRJPXJ3CIGEXCVDIAPA5GVOC2PWQ, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.436547109 (58h2m2.159110149s) [noise=true , address=104.36.82.41:4785 , piece_id=6Y7NIN5ZWOPWLRVRQDDIR3MXUKFUFLJVZPSNYGTICOKBD4VNXENA, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.436794893 (58h2m2.159357933s) [noise=true , address=79.127.201.215:4357 , piece_id=6FH4IK3DJA2YUBOG2QKZSLMUZQ5IZJK2Z7K7AKJ6GGL4SSSTROZQ, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.437029562 (58h2m2.159592602s) [noise=true , address=104.36.82.8:4983 , piece_id=JZWTQLKXWRB2JYR4QL7XHC3ELKJ6MDNFY37OGMLGTK7YN6S3XUWQ, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:52.437335538 (58h2m2.159898578s) [noise=true , address=104.36.82.10:4086 , piece_id=CYG5TUFDRBZE4MLWWLWL5HXT42TXVOEWRYK3UAB62JQHWT5UOTBA, seg_pos={0 0}, upload=55701]: Uploading piece... Jul 26 21:37:53.536902411 (58h2m3.259465451s) [upload=55701]: Next returned nil segment Jul 26 21:37:53.536919163 (58h2m3.259482193s) [upload=55701]: Waiting for error group managing segments... Jul 26 21:37:53.846129711 (58h2m3.568692751s) [noise=true , address=79.127.205.233:4639 , piece_id=MGJWIUAGQETAWJHPXT52BUL5HEFXJONHJDOYYVX6RPKODGHUX3EQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:37:53.912793631 (58h2m3.635356671s) [noise=true , address=79.127.205.234:4612 , piece_id=YGPM7PFBTHUDM5EN5OZBXWBDAMBIQM6SG5EV2UIGVGSTPZLPWW3A, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:37:54.170969113 (58h2m3.893532153s) [noise=true , address=79.127.220.101:4048 , piece_id=UII2VDUXPLXZTELVTOYSSU6G7DDZIXNVH24IDXJS42XTOCNTH62Q, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:37:54.314191844 (58h2m4.036754885s) [noise=true , address=79.127.219.38:4013 , piece_id=57CUI4Q5CINFQEBOKIN4FEHHTSPDTVAI7YUSSRVFKREBZBC3Z4TQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:37:54.329460908 (58h2m4.052023949s) [noise=true , address=45.91.116.10:4111 , piece_id=7SIGL2STGEMEJFACJXWADLA5JRJPXJ3CIGEXCVDIAPA5GVOC2PWQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:37:54.981418592 (58h2m4.703981632s) [noise=true , address=79.127.201.215:4357 , piece_id=6FH4IK3DJA2YUBOG2QKZSLMUZQ5IZJK2Z7K7AKJ6GGL4SSSTROZQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:37:57.970075790 (58h2m7.69263883s) [noise=true , address=45.91.116.14:4222 , piece_id=UXEKJAXAOEILZT6ENTDKQMJUAVS2BQ2HTA6B5TDVAGUUCOCIOD3A, seg_pos={0 0}, upload=55701]: Done uploading piece. err: <nil> Jul 26 21:38:51.351703464 (58h3m1.074266504s) [noise=true , address=104.36.82.49:4977 , piece_id=GZP6553R6AP5GKDNDREIWOJ2KAYJTTDUHTY2Y5LKALDIVOLKKYGQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:104.36.82.49:4977): protocol: write tcp 188.166.225.170:60564->104.36.82.49:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351754602 (58h3m1.074317642s) [noise=true , address=104.36.82.8:4983 , piece_id=JZWTQLKXWRB2JYR4QL7XHC3ELKJ6MDNFY37OGMLGTK7YN6S3XUWQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:12h6MND3rBadG4EjUP18S7izyj23cVBLaB5drFsuNGhcxwAYXSA, address:104.36.82.8:4983): protocol: write tcp 188.166.225.170:40006->104.36.82.8:4983: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351789058 (58h3m1.074352098s) [noise=true , address=104.36.82.10:4086 , piece_id=CYG5TUFDRBZE4MLWWLWL5HXT42TXVOEWRYK3UAB62JQHWT5UOTBA, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:1fMyQ9ojAvrZhG4sLwG3wzNJZihg8ZwWKBJmgWpK3Ckx5YXrg3, address:104.36.82.10:4086): protocol: write tcp 188.166.225.170:36354->104.36.82.10:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351823213 (58h3m1.074386253s) [noise=true , address=104.36.82.55:4206 , piece_id=7B2GUPV4KHS2252G4GEBMLAIA23VO37FXOZDCRZZXRHPTCNLK7SA, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:104.36.82.55:4206): protocol: write tcp 188.166.225.170:44348->104.36.82.55:4206: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351830857 (58h3m1.074393938s) [noise=true , address=104.36.82.54:4170 , piece_id=FFJTYTCJX32QIVS2LKYZHIWNNM35FJZU6F5S4WPZGGXVQ37TF42A, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:1mcYgUVVeD2xo4pDhR7ywPLRgyA7GgejeErtLiWiPTV2AhbBRX, address:104.36.82.54:4170): protocol: write tcp 188.166.225.170:52798->104.36.82.54:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351853441 (58h3m1.074416481s) [noise=true , address=104.36.82.56:4240 , piece_id=HZJ3DMH6WLLTIVOSRMHBVU6O3B7GYFOLSEEZAMU4YBPQ4OE67X4A, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:129QqkFEzZdtzQ6HAKxPWxDNgDVUNJJLH59oRVXnHExrwkDECyf, address:104.36.82.56:4240): protocol: write tcp 188.166.225.170:38566->104.36.82.56:4240: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351885482 (58h3m1.074448522s) [noise=true , address=104.36.82.52:4102 , piece_id=JBZ4R5KAX2O5VKSM2P4OW6XMTS77IK6DNWOTGOU4EK7XSLOX66TQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:12evG7WqJ9XXqh2z2kjGr4Wz1q5mrcySfTMczqEm7i8jgYsW8qP, address:104.36.82.52:4102): protocol: write tcp 188.166.225.170:44004->104.36.82.52:4102: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351913355 (58h3m1.074476395s) [noise=true , address=104.36.82.46:4492 , piece_id=CBQI2NY5XCSJ3EONPVV3VLBATEFV2E3MXM4QLICCBVRA5QH4UHWQ, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:1roa5GGYmqCX2AxSrqFgqyV2CtBEYZavuGTseiwDy4pjkT6bip, address:104.36.82.46:4492): protocol: write tcp 188.166.225.170:37642->104.36.82.46:4492: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351916782 (58h3m1.074479842s) [noise=true , address=104.36.82.3:4780 , piece_id=63JWNPKTNIFCEM4J47S4YNAQGNPJZNMFJ654RPOWN43Q6LKSNFSA, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM, address:104.36.82.3:4780): protocol: write tcp 188.166.225.170:34744->104.36.82.3:4780: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351932963 (58h3m1.074496003s) [noise=true , address=104.36.82.53:4130 , piece_id=XOVNSIS3BQS5JQHMQPKP6MDV6M4XOLISJRYXESJPUG32FVBUSI3Q, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:12N8MGboFyEV381rkrBva3G3advz3QQEY2HCHxwqzKX2X5zxNvb, address:104.36.82.53:4130): protocol: write tcp 188.166.225.170:59702->104.36.82.53:4130: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351939836 (58h3m1.074502876s) [noise=true , address=104.36.82.41:4785 , piece_id=6Y7NIN5ZWOPWLRVRQDDIR3MXUKFUFLJVZPSNYGTICOKBD4VNXENA, seg_pos={0 0}, upload=55701]: Done uploading piece. err: ecclient: upload failed (node:1gYH83ECV8wai3CZyBSU8NNJM1UsysLYHbQDwBVKZRyjsdnUwB, address:104.36.82.41:4785): protocol: write tcp 188.166.225.170:41688->104.36.82.41:4785: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:51.351953222 (58h3m1.074516262s) [seg_pos={0 0}, upload=55701]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 26 21:38:51.580766265 (58h3m1.303329305s) [upload=55701]: Done uploading segments... Jul 26 21:38:51.580800030 (58h3m1.3033631s) [upload=55701]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.231222011s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X....X......X] duration: 30.229123293s FAIL (2025-07-26 21:38:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== lose: conn closed: op Jul 26 21:37:56.752785726 (58h2m6.475348756s) [noise=true , address=104.36.82.10:4079 , piece_id=YN3NDUBMW4XKWOWKDEXDDCJEHNOZPT7VKJTETOYUPIADJDWTUQKA, seg_pos={0 0}, upload=55698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: context canceled; context canceled Jul 26 21:37:56.752804933 (58h2m6.475367973s) [seg_pos={0 0}, upload=55698]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:37:56.752868103 (58h2m6.475431143s) [upload=55698]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:37:56.752872151 (58h2m6.475435181s) [upload=55698]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:37:57.041780876 (58h2m6.764343946s) [upload=55698]: Done uploading segments... Jul 26 21:37:57.041833446 (58h2m6.764396496s) [upload=55698]: Upload finished. err: <nil> Jul 26 21:38:21.249244821 (58h2m30.971807851s) [upload=55710]: Starting upload Jul 26 21:38:21.249338500 (58h2m30.97190153s) [upload=55710]: Done starting upload Jul 26 21:38:21.249593788 (58h2m30.972156828s) [upload=55710]: Uploading segments... Jul 26 21:38:21.249764665 (58h2m30.972327705s) [upload=55710]: Got next segment. Inline: false Jul 26 21:38:21.249793581 (58h2m30.972356611s) [upload=55710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:21.249833547 (58h2m30.972396577s) [upload=55710]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:21.697835017 (58h2m31.420398057s) [seg_pos={0 0}, upload=55710]: Begin upload segment... Jul 26 21:38:21.697877459 (58h2m31.420440499s) [seg_pos={0 0}, upload=55710]: Done begin upload segment. Jul 26 21:38:21.697899199 (58h2m31.420462239s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.698034599 (58h2m31.420597629s) [noise=true , address=77.90.31.16:4735 , piece_id=KLRE5IIMPPVDN236Z4WVOU3SID5D5KQJGYMQSCW26SCSPNIOLGKQ, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.698328441 (58h2m31.420891511s) [noise=true , address=45.91.116.12:4227 , piece_id=NN3QANK7N6LGG6LBIGIC5K7KYFMVWPRUQLOEKH477UIRUH4YBO6A, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.698543302 (58h2m31.421106342s) [noise=true , address=77.90.31.18:4806 , piece_id=CAVRVJJMHCVPRPNTQOZBPWXELWZG5JWGQUZPMX3VF6Q4LHWLEM3A, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.698634126 (58h2m31.421197166s) [noise=true , address=45.91.116.11:4117 , piece_id=CBDFMPF3XFPODSY6PBKLK7QE2TS3ZFBYTAVY45IFQAGADFIEGQWQ, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.698958707 (58h2m31.421521747s) [noise=true , address=104.36.82.16:4295 , piece_id=35GDEASLEL7GMD3KBDO2OTF3H2FJR5NGPVYLFAJVYLL2FEPVTVWQ, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699004014 (58h2m31.421567054s) [noise=true , address=45.91.116.14:4191 , piece_id=OA2XC7UHFXY7P2TWH3Y44ZAFFENZIHFCNIZDKY5V2IKHXY4KBXEA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699281685 (58h2m31.421844725s) [noise=true , address=45.91.116.6:4974 , piece_id=P442RFIN27BEEXLE2PRAEQQ2CD5ES5D5QMZOPGKJ7YAO5TLMTG7Q, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699319167 (58h2m31.421882207s) [noise=true , address=104.36.82.7:4910 , piece_id=NWIFYGKNWPV6CRQUTAVKUSAEHETYRMUNTT7YPU73Z5EVHVEX67WA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699568253 (58h2m31.422131293s) [noise=true , address=45.91.116.8:4047 , piece_id=TDB3RXD7Z4XIYLT3HHW57CSWAWHB27CU5AAUYACNTRGKHZNZZVEQ, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699643367 (58h2m31.422206397s) [noise=true , address=77.90.31.15:4698 , piece_id=KE6AXEBE4LBY6ZSFNG5AS5FOKGYI7FKLTRUIZRPUSQKGIOSXK4XA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699806559 (58h2m31.422369599s) [noise=true , address=45.91.116.6:4973 , piece_id=Z4V3BZ6UZHGEFYWKCKIHM77OBKJTGUBGEUS5XTY7BZR7JBE4H5UQ, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.699955424 (58h2m31.422518454s) [noise=true , address=45.91.116.5:4297 , piece_id=CAQ4HLNELHNPZ7QDLSK6NLCBZHR6CEG4MIGNKNGFXW5DE3YNWGKQ, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.700053382 (58h2m31.422616422s) [noise=true , address=104.36.82.14:4221 , piece_id=O4F5TYRMITC7RTVD24Z626GKOIVO3A4HYWFP4N52PU2KCH3YHTAA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.700286708 (58h2m31.422849748s) [noise=true , address=45.91.116.10:4082 , piece_id=HNFKN75WLOWN4A65GKOVUG3VZ63MO7B3MQQH4CTBBOHTZ2JCPJAA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:21.700366070 (58h2m31.42292911s) [noise=true , address=77.90.31.10:4518 , piece_id=7ALLQWUCR5ACQDKSCYRXQ5OC3CQVUZJ3KOJOUQYX4FWNE2KRDXWA, seg_pos={0 0}, upload=55710]: Uploading piece... Jul 26 21:38:23.430974418 (58h2m33.153537458s) [upload=55710]: Next returned nil segment Jul 26 21:38:23.430996570 (58h2m33.15355961s) [upload=55710]: Waiting for error group managing segments... Jul 26 21:38:23.706145357 (58h2m33.428708398s) [noise=true , address=45.91.116.8:4047 , piece_id=TDB3RXD7Z4XIYLT3HHW57CSWAWHB27CU5AAUYACNTRGKHZNZZVEQ, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.722288895 (58h2m33.444851935s) [noise=true , address=45.91.116.12:4227 , piece_id=NN3QANK7N6LGG6LBIGIC5K7KYFMVWPRUQLOEKH477UIRUH4YBO6A, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.724543299 (58h2m33.447106339s) [noise=true , address=45.91.116.10:4082 , piece_id=HNFKN75WLOWN4A65GKOVUG3VZ63MO7B3MQQH4CTBBOHTZ2JCPJAA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.778556774 (58h2m33.501119814s) [noise=true , address=45.91.116.14:4191 , piece_id=OA2XC7UHFXY7P2TWH3Y44ZAFFENZIHFCNIZDKY5V2IKHXY4KBXEA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.815169305 (58h2m33.537732345s) [noise=true , address=45.91.116.11:4117 , piece_id=CBDFMPF3XFPODSY6PBKLK7QE2TS3ZFBYTAVY45IFQAGADFIEGQWQ, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.824110092 (58h2m33.546673122s) [noise=true , address=45.91.116.5:4297 , piece_id=CAQ4HLNELHNPZ7QDLSK6NLCBZHR6CEG4MIGNKNGFXW5DE3YNWGKQ, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.866715560 (58h2m33.5892786s) [noise=true , address=45.91.116.6:4974 , piece_id=P442RFIN27BEEXLE2PRAEQQ2CD5ES5D5QMZOPGKJ7YAO5TLMTG7Q, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:23.901856205 (58h2m33.624419245s) [noise=true , address=45.91.116.6:4973 , piece_id=Z4V3BZ6UZHGEFYWKCKIHM77OBKJTGUBGEUS5XTY7BZR7JBE4H5UQ, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:27.008844909 (58h2m36.731407949s) [noise=true , address=77.90.31.18:4806 , piece_id=CAVRVJJMHCVPRPNTQOZBPWXELWZG5JWGQUZPMX3VF6Q4LHWLEM3A, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:27.904894077 (58h2m37.627457107s) [noise=true , address=77.90.31.15:4698 , piece_id=KE6AXEBE4LBY6ZSFNG5AS5FOKGYI7FKLTRUIZRPUSQKGIOSXK4XA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:28.884906405 (58h2m38.607469445s) [noise=true , address=77.90.31.16:4735 , piece_id=KLRE5IIMPPVDN236Z4WVOU3SID5D5KQJGYMQSCW26SCSPNIOLGKQ, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:29.953084712 (58h2m39.675647752s) [noise=true , address=77.90.31.10:4518 , piece_id=7ALLQWUCR5ACQDKSCYRXQ5OC3CQVUZJ3KOJOUQYX4FWNE2KRDXWA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: <nil> Jul 26 21:38:50.605659845 (58h3m0.328222885s) [noise=true , address=104.36.82.16:4295 , piece_id=35GDEASLEL7GMD3KBDO2OTF3H2FJR5NGPVYLFAJVYLL2FEPVTVWQ, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 188.166.225.170:59774->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:50.605705952 (58h3m0.328268992s) [noise=true , address=104.36.82.7:4910 , piece_id=NWIFYGKNWPV6CRQUTAVKUSAEHETYRMUNTT7YPU73Z5EVHVEX67WA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: write tcp 188.166.225.170:44812->104.36.82.7:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:50.605863905 (58h3m0.328426945s) [noise=true , address=104.36.82.14:4221 , piece_id=O4F5TYRMITC7RTVD24Z626GKOIVO3A4HYWFP4N52PU2KCH3YHTAA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 188.166.225.170:40774->104.36.82.14:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:38:50.605925663 (58h3m0.328488703s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:38:50.605944900 (58h3m0.32850794s) [seg_pos={0 0}, upload=55710]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:50.833093218 (58h3m0.555656258s) [upload=55710]: Done uploading segments... Jul 26 21:38:50.833138233 (58h3m0.555701293s) [upload=55710]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.229143984s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X...................................XX......] duration: 20.375416068s FAIL (2025-07-26 21:38:49 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 20.375434754s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.........X.................X.....] duration: 1.886015612s FAIL (2025-07-26 21:38:07 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.886043205s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X....X......] duration: 14.952772865s FAIL (2025-07-26 21:38:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 14.952798925s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 1m0.228654748s FAIL (2025-07-26 21:37:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== .82.18:4380 , piece_id=UHAVNOCTPKVZHOMDQGAMAJYFWWTK2TXY4FLZBJGSXAD6GLL2Y4YQ, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.985028150 (58h1m1.70759119s) [noise=true , address=104.36.82.54:4185 , piece_id=PMVEKDJ6TMWGGXR3I4HBJ5UF63JX3EVU5G7K6SJW2D7ZU6ONAUSA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.985237181 (58h1m1.707800211s) [noise=true , address=79.127.219.48:4084 , piece_id=D66E6JICDQYLZT3G6VLY7PK4D3U422DVIBSR2NX3SEQ5KLJG6BWQ, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.985444687 (58h1m1.708007727s) [noise=true , address=104.36.82.46:4493 , piece_id=YBFHFEOX7NEPKKRNLLLXLIU5HAFNI2NTY6EVSVA7B4ZHQNCZDPVA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.985661432 (58h1m1.708224462s) [noise=true , address=79.127.219.38:4006 , piece_id=DJMDXUO73CUSOAGMBKFFMBYOJGHZJ27DY6HAS5NP7AQ6252RONXA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.985915869 (58h1m1.708478909s) [noise=true , address=104.36.82.41:4786 , piece_id=GCNXGIQWM6JJIWJSTKP2CVY7Z2EQBRCICIFWPDI3NLCDEEN7RO4Q, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.986133215 (58h1m1.708696255s) [noise=true , address=104.36.82.50:4024 , piece_id=B7R4MDNMQXTLLIU3MPGAKC7V7REVDXWKTRAASLXWI7FIDORKIHGQ, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.986341213 (58h1m1.708904253s) [noise=true , address=104.36.82.20:4464 , piece_id=ZCGTPM5OKXNEQ6SI4BPB5HIN4PEZ3XRREBX6RSAKNVBSX2AOOAUA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.986523843 (58h1m1.709086873s) [noise=true , address=79.127.201.215:4358 , piece_id=4T5JFA4GQPUWUUJOJ6YSRALU5ECGDVMSLNSAWLHH5PN4IVCRFTZA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.986706862 (58h1m1.709269902s) [noise=true , address=104.36.82.53:4140 , piece_id=73UTM2RV4JP5JTJ3Z3M555IDJXTE4GX4ITRJQ7BQOG43ORSUFDHA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.986894271 (58h1m1.709457311s) [noise=true , address=79.127.205.234:4604 , piece_id=KHEY5AMK3FASFGBLU5YBFJGBGXZ3GPY7IDIAYAWANETVGBUK7AIA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.987082611 (58h1m1.709645652s) [noise=true , address=104.36.82.6:4885 , piece_id=YNBVQHMONC4KB5IRBXP3WN4J2HN3RVYS3BUTRWZMSHIT36OPYJMA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.987257155 (58h1m1.709820195s) [noise=true , address=104.36.82.56:4248 , piece_id=VU6RBTNE23OLU4ELZD5OCSICHCRZJS3LDRXJH7OTJJCECYNIZWIA, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:51.987432571 (58h1m1.709995611s) [noise=true , address=104.36.82.36:4638 , piece_id=CZGPXDSUAWSJTRN5WGZDX4AGKIPVZVZXZXNN5DNMQ47T26OBLBJQ, seg_pos={0 0}, upload=55685]: Uploading piece... Jul 26 21:36:53.100659211 (58h1m2.823222251s) [upload=55685]: Next returned nil segment Jul 26 21:36:53.100678538 (58h1m2.823241579s) [upload=55685]: Waiting for error group managing segments... Jul 26 21:36:53.449910250 (58h1m3.17247329s) [noise=true , address=79.127.205.233:4646 , piece_id=BV7U54FGVUKSY4JRWUXPLFPANYQYY4URQMYCLJC6LCL3NGSB5GYA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: <nil> Jul 26 21:36:53.793559952 (58h1m3.516122992s) [noise=true , address=79.127.205.234:4604 , piece_id=KHEY5AMK3FASFGBLU5YBFJGBGXZ3GPY7IDIAYAWANETVGBUK7AIA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: <nil> Jul 26 21:36:53.955250069 (58h1m3.677813109s) [noise=true , address=79.127.201.215:4358 , piece_id=4T5JFA4GQPUWUUJOJ6YSRALU5ECGDVMSLNSAWLHH5PN4IVCRFTZA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: <nil> Jul 26 21:36:54.052956422 (58h1m3.775519462s) [noise=true , address=79.127.219.48:4084 , piece_id=D66E6JICDQYLZT3G6VLY7PK4D3U422DVIBSR2NX3SEQ5KLJG6BWQ, seg_pos={0 0}, upload=55685]: Done uploading piece. err: <nil> Jul 26 21:36:54.101010645 (58h1m3.823573685s) [noise=true , address=79.127.219.38:4006 , piece_id=DJMDXUO73CUSOAGMBKFFMBYOJGHZJ27DY6HAS5NP7AQ6252RONXA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: <nil> Jul 26 21:36:54.168324898 (58h1m3.890887928s) [noise=true , address=45.91.116.8:4055 , piece_id=I5X2JHCGBRSSIXF4UQAZ6YKQX62BFCRYCAMTR5FT5KIJEGGBQ3ZA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: <nil> Jul 26 21:37:50.886058677 (58h2m0.608621717s) [noise=true , address=104.36.82.18:4380 , piece_id=UHAVNOCTPKVZHOMDQGAMAJYFWWTK2TXY4FLZBJGSXAD6GLL2Y4YQ, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1Ebyj96KWVGKwgzRjkAWc9ktcaRW5xqq1bJLEPxfepNvzWvyWN, address:104.36.82.18:4380): protocol: rpc: dial tcp 104.36.82.18:4380: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886109635 (58h2m0.608672675s) [noise=true , address=104.36.82.50:4024 , piece_id=B7R4MDNMQXTLLIU3MPGAKC7V7REVDXWKTRAASLXWI7FIDORKIHGQ, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1EAeBfX3Xyp8pBDUoS7VhyXwpNbXooSYgxZPxP9kEXT1rZwkoG, address:104.36.82.50:4024): protocol: rpc: dial tcp 104.36.82.50:4024: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886141856 (58h2m0.608704896s) [noise=true , address=104.36.82.20:4464 , piece_id=ZCGTPM5OKXNEQ6SI4BPB5HIN4PEZ3XRREBX6RSAKNVBSX2AOOAUA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1xKikZTtJ6dr1ry8oAkLc7fmyB51jydk87BWoxuTTzKFDeToEY, address:104.36.82.20:4464): protocol: rpc: dial tcp 104.36.82.20:4464: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886173046 (58h2m0.608736086s) [noise=true , address=104.36.82.56:4248 , piece_id=VU6RBTNE23OLU4ELZD5OCSICHCRZJS3LDRXJH7OTJJCECYNIZWIA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1QpYBQ2PfiAwvt5EmFvjBnKgBhwnuyF6b77qJocpNe1YqSzP77, address:104.36.82.56:4248): protocol: write tcp 188.166.225.170:56414->104.36.82.56:4248: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886252257 (58h2m0.608815287s) [noise=true , address=104.36.82.47:4533 , piece_id=3F245HU2UCF6SUIZOIMYHHXALXQ4IOBHTF2XQ7U5CR4JZFON76WQ, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR, address:104.36.82.47:4533): protocol: rpc: dial tcp 104.36.82.47:4533: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886288476 (58h2m0.608851506s) [noise=true , address=104.36.82.54:4185 , piece_id=PMVEKDJ6TMWGGXR3I4HBJ5UF63JX3EVU5G7K6SJW2D7ZU6ONAUSA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1cLdyyVLrp1yBmgsfBrfYc6N2RFbBichYoBbo2F7EZfaBXtkfk, address:104.36.82.54:4185): protocol: rpc: dial tcp 104.36.82.54:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886317662 (58h2m0.608880702s) [noise=true , address=104.36.82.41:4786 , piece_id=GCNXGIQWM6JJIWJSTKP2CVY7Z2EQBRCICIFWPDI3NLCDEEN7RO4Q, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:125TWU7R4pkmrUxsNLhgTD475aLbjUJ7psA6drJtE7oDqZXYJ4Q, address:104.36.82.41:4786): protocol: rpc: dial tcp 104.36.82.41:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886365103 (58h2m0.608928143s) [noise=true , address=104.36.82.53:4140 , piece_id=73UTM2RV4JP5JTJ3Z3M555IDJXTE4GX4ITRJQ7BQOG43ORSUFDHA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:12Wv1U4bCkzg9Ywx4EKRkWG8tMffjwiVBEZ6tHr8scNHfdcWyrL, address:104.36.82.53:4140): protocol: rpc: dial tcp 104.36.82.53:4140: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886392716 (58h2m0.608955746s) [noise=true , address=104.36.82.6:4885 , piece_id=YNBVQHMONC4KB5IRBXP3WN4J2HN3RVYS3BUTRWZMSHIT36OPYJMA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:12rLd1AWyRtEVFmK4iJCebp5VAJEjnmhhgtUQcV1FB8vMvRnUhm, address:104.36.82.6:4885): protocol: rpc: dial tcp 104.36.82.6:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886419487 (58h2m0.608982527s) [noise=true , address=104.36.82.36:4638 , piece_id=CZGPXDSUAWSJTRN5WGZDX4AGKIPVZVZXZXNN5DNMQ47T26OBLBJQ, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1poiPDhUGb5ZGnZnZcRViY45RefT9anD83YFcqZVZspnnJ9fN5, address:104.36.82.36:4638): protocol: rpc: dial tcp 104.36.82.36:4638: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886426641 (58h2m0.608989681s) [noise=true , address=104.36.82.46:4493 , piece_id=YBFHFEOX7NEPKKRNLLLXLIU5HAFNI2NTY6EVSVA7B4ZHQNCZDPVA, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:127drPYZWm7RRXuCLtuzDgSaxxuu3k1tg6eo3KWtrQGjBaUbfRJ, address:104.36.82.46:4493): protocol: write tcp 188.166.225.170:34414->104.36.82.46:4493: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886448773 (58h2m0.609011803s) [noise=true , address=104.36.82.52:4098 , piece_id=A75P4EARF2NUEY3GM2ZUYLQXCWY7YPTDKJ3P4IZVSCSH5QOV6T4Q, seg_pos={0 0}, upload=55685]: Done uploading piece. err: ecclient: upload failed (node:1SoFNdwfh4p5tXUkpfjQaev6tdA2ZdhsXUFvZNBsy5QrJ9zdqb, address:104.36.82.52:4098): protocol: rpc: dial tcp 104.36.82.52:4098: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:37:50.886468501 (58h2m0.609031541s) [seg_pos={0 0}, upload=55685]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:37:51.113340050 (58h2m0.83590309s) [upload=55685]: Done uploading segments... Jul 26 21:37:51.113365288 (58h2m0.835928328s) [upload=55685]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.2286666s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.........X.................X....] duration: 1.879424843s FAIL (2025-07-26 21:37:50 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.879451894s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X...................................XX.....] duration: 6.121826125s FAIL (2025-07-26 21:37:22 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.121848607s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............................X.........X.................X...] duration: 1.852292308s FAIL (2025-07-26 21:37:21 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.852317837s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X....X.....] duration: 21.178939128s FAIL (2025-07-26 21:36:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 21.1789613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 1m0.234647084s FAIL (2025-07-26 21:36:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== ece_id=OXDEYX4JVDHJMUIKDUZUH5Z4WHGMLKHBEJSHDWXXTDDBDECPUTSQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.519185775 (58h0m1.241748805s) [noise=true , address=104.36.82.56:4238 , piece_id=JPRGR44RUSP23YXAM62DLCE3L4AKZNZDNY3JYPQTEDYAX3EBKTXQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.519530374 (58h0m1.242093414s) [noise=true , address=64.49.13.139:4474 , piece_id=SD2HS5XGE4HPP6J4QUFCEPCR6HBJV32PKVIGIT24ERM5W7EQDCXA, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.519542448 (58h0m1.242105488s) [noise=true , address=79.127.219.51:4212 , piece_id=7YVZTTNXL46VSSBTJCWSM4642Q6WUFFNXDVN2VW4ABCOVZAFOJSA, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.519863321 (58h0m1.242426361s) [noise=true , address=104.36.82.42:4803 , piece_id=A7Y45V6CPMMVXOCGVPIGY3VPO6KEJKRGYAD77I4X357J65KOTQQQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.519900723 (58h0m1.242463753s) [noise=true , address=156.146.43.225:4434 , piece_id=YYA7GZXFQCQFB7IFD2E7Y7N7XGAGWBG3DMEHK5QR74CEPOB2QONA, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.520196508 (58h0m1.242759548s) [noise=true , address=104.36.82.19:4432 , piece_id=3ST5R55YXHGLS65WNCO6GDPSB4GHK5ZZ7QONK36TQWKRXDW7KTSQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.520255061 (58h0m1.242818101s) [noise=true , address=79.127.205.234:4607 , piece_id=LTH6JYTHACYPSXFD4PSJIHQXUQD5UOF7SIEJH5Y2E3YBSA7KIROQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.520598057 (58h0m1.243161087s) [noise=true , address=104.36.82.13:4209 , piece_id=6KITLEIRKDUZ6IIQVV35BRX7VHPQJJVCWTVZUB77KDHTISH2LEBQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.520624898 (58h0m1.243187928s) [noise=true , address=79.127.205.245:4242 , piece_id=5BXQG4LLANPJMQAPMIQ6KGY5TPDNDWFHWVESWR67DVJSOQTIHMEQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.520873384 (58h0m1.243436424s) [noise=true , address=104.36.82.53:4131 , piece_id=ZS3WYGNN5QDJFRE3ZFW7E7YAM26NQY43IGFEUUK27TIESN5NBV2A, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.520913491 (58h0m1.243476531s) [noise=true , address=104.36.82.50:4042 , piece_id=HHXYPCBE7VAP2M5H676O4N7WYUAX3KGIHDCP352RTXYJCINGMRBQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.521174700 (58h0m1.24373774s) [noise=true , address=104.36.82.54:4159 , piece_id=QWQ7VZTX5A4BKDXN423QTUQAU2BETAELY2LPX5PK3UYW46EID4OQ, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.521461068 (58h0m1.244024108s) [noise=true , address=77.90.31.21:4912 , piece_id=ZCFWQHUCDYA37YLVXRADNDWTQ4KRUS5ELYE5YBF4BLLEABPNZZTA, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:51.521483291 (58h0m1.244046331s) [noise=true , address=104.36.82.21:4503 , piece_id=TXH7GOJM3CH65MKK53USYKNJA426BIUHR2ZF73QRA7QRDDGWFPSA, seg_pos={0 0}, upload=55669]: Uploading piece... Jul 26 21:35:52.648335540 (58h0m2.37089857s) [upload=55669]: Next returned nil segment Jul 26 21:35:52.648359325 (58h0m2.370922366s) [upload=55669]: Waiting for error group managing segments... Jul 26 21:35:52.933425228 (58h0m2.655988258s) [noise=true , address=79.127.205.234:4607 , piece_id=LTH6JYTHACYPSXFD4PSJIHQXUQD5UOF7SIEJH5Y2E3YBSA7KIROQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:35:53.030086070 (58h0m2.75264911s) [noise=true , address=64.49.13.139:4474 , piece_id=SD2HS5XGE4HPP6J4QUFCEPCR6HBJV32PKVIGIT24ERM5W7EQDCXA, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:35:53.088381729 (58h0m2.810944769s) [noise=true , address=79.127.205.245:4242 , piece_id=5BXQG4LLANPJMQAPMIQ6KGY5TPDNDWFHWVESWR67DVJSOQTIHMEQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:35:53.550582848 (58h0m3.273145918s) [noise=true , address=79.127.219.51:4212 , piece_id=7YVZTTNXL46VSSBTJCWSM4642Q6WUFFNXDVN2VW4ABCOVZAFOJSA, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:35:53.550673972 (58h0m3.273237012s) [noise=true , address=45.91.116.10:4111 , piece_id=HTHSBRZVLVT5ZSCYAACG3OAQDDATOZBSZ6XP5GYVE3BF3KSK2JOA, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:35:55.096506986 (58h0m4.819070046s) [noise=true , address=156.146.43.225:4434 , piece_id=YYA7GZXFQCQFB7IFD2E7Y7N7XGAGWBG3DMEHK5QR74CEPOB2QONA, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:35:57.894177014 (58h0m7.616740054s) [noise=true , address=77.90.31.21:4912 , piece_id=ZCFWQHUCDYA37YLVXRADNDWTQ4KRUS5ELYE5YBF4BLLEABPNZZTA, seg_pos={0 0}, upload=55669]: Done uploading piece. err: <nil> Jul 26 21:36:50.422400053 (58h1m0.144963093s) [noise=true , address=104.36.82.5:4848 , piece_id=OXDEYX4JVDHJMUIKDUZUH5Z4WHGMLKHBEJSHDWXXTDDBDECPUTSQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:1xvod7N8eEwunLDYWAvVUP1Cp1CdEr3quJUMG5kWyBrwzzr9vp, address:104.36.82.5:4848): protocol: rpc: dial tcp 104.36.82.5:4848: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422447393 (58h1m0.145010423s) [noise=true , address=104.36.82.19:4432 , piece_id=3ST5R55YXHGLS65WNCO6GDPSB4GHK5ZZ7QONK36TQWKRXDW7KTSQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:12oh9XZaiLhaZdMyq1cx88yrrG9C9sHtH3RTA6W2Ly39t23ArSQ, address:104.36.82.19:4432): protocol: rpc: dial tcp 104.36.82.19:4432: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422469635 (58h1m0.145032675s) [noise=true , address=104.36.82.52:4095 , piece_id=DGP6DEGJYMYHMYPBAAYD4MRPQWYPTNLKUACJTPGJSEM3GB7I6PJQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:1idvNc8SGRP7tkAckr2TLv7LnvftiRUTGcwy5GCrH5rVyDEK5N, address:104.36.82.52:4095): protocol: write tcp 188.166.225.170:39318->104.36.82.52:4095: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422482000 (58h1m0.14504504s) [noise=true , address=104.36.82.53:4131 , piece_id=ZS3WYGNN5QDJFRE3ZFW7E7YAM26NQY43IGFEUUK27TIESN5NBV2A, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:1uhyc5ozrF9ndeEnQXtjc9o3tsYiAnW1dzeuHV3kv1JS4T4a8C, address:104.36.82.53:4131): protocol: rpc: dial tcp 104.36.82.53:4131: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422514792 (58h1m0.145077832s) [noise=true , address=104.36.82.54:4159 , piece_id=QWQ7VZTX5A4BKDXN423QTUQAU2BETAELY2LPX5PK3UYW46EID4OQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA, address:104.36.82.54:4159): protocol: rpc: dial tcp 104.36.82.54:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422533157 (58h1m0.145096197s) [noise=true , address=104.36.82.13:4209 , piece_id=6KITLEIRKDUZ6IIQVV35BRX7VHPQJJVCWTVZUB77KDHTISH2LEBQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:18W2j3VhAMCmWAuST1M1yLgCWBZP3hQwGkLUvFAMSzUmhLpVRc, address:104.36.82.13:4209): protocol: write tcp 188.166.225.170:40100->104.36.82.13:4209: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422569817 (58h1m0.145132888s) [noise=true , address=104.36.82.21:4503 , piece_id=TXH7GOJM3CH65MKK53USYKNJA426BIUHR2ZF73QRA7QRDDGWFPSA, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:17F3Gn5LYgekp8WgagEdFXtRu8jCc27qA9UuW7pWsJ9FUFKFSi, address:104.36.82.21:4503): protocol: rpc: dial tcp 104.36.82.21:4503: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422584465 (58h1m0.145147505s) [noise=true , address=104.36.82.50:4042 , piece_id=HHXYPCBE7VAP2M5H676O4N7WYUAX3KGIHDCP352RTXYJCINGMRBQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:1LbRrZsqp1TR354jZQLstDmWAUBVBMyid8AuAbTfQj7N89Mnte, address:104.36.82.50:4042): protocol: rpc: dial tcp 104.36.82.50:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422627027 (58h1m0.145190077s) [noise=true , address=104.36.82.44:4896 , piece_id=KZWPSMKYZJZ5NUKDVS6W2CU4JW4Y6AUK2YABDT27EMO7HTRDIOFQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o, address:104.36.82.44:4896): protocol: rpc: dial tcp 104.36.82.44:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422635182 (58h1m0.145198222s) [noise=true , address=104.36.82.56:4238 , piece_id=JPRGR44RUSP23YXAM62DLCE3L4AKZNZDNY3JYPQTEDYAX3EBKTXQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5, address:104.36.82.56:4238): protocol: rpc: dial tcp 104.36.82.56:4238: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422670770 (58h1m0.14523381s) [noise=true , address=104.36.82.42:4803 , piece_id=A7Y45V6CPMMVXOCGVPIGY3VPO6KEJKRGYAD77I4X357J65KOTQQQ, seg_pos={0 0}, upload=55669]: Done uploading piece. err: ecclient: upload failed (node:12GFk3GkhB88UKQdZM8vZ9VLNuuYDrS5ThZjVAdbL2CtTQVMDEB, address:104.36.82.42:4803): protocol: write tcp 188.166.225.170:38132->104.36.82.42:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:36:50.422701549 (58h1m0.145264629s) [seg_pos={0 0}, upload=55669]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 26 21:36:50.655782107 (58h1m0.378345147s) [upload=55669]: Done uploading segments... Jul 26 21:36:50.655805892 (58h1m0.378368922s) [upload=55669]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.234657493s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.........X.................X..] duration: 1.84502826s FAIL (2025-07-26 21:36:42 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.845050964s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X...................................XX....] duration: 11.596781244s FAIL (2025-07-26 21:36:37 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 11.596806211s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.........X.................X.] duration: 1.863035733s FAIL (2025-07-26 21:36:00 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.863099835s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X...................................XX...] duration: 12.62272322s FAIL (2025-07-26 21:35:54 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 12.622773868s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 1m0.231842322s FAIL (2025-07-26 21:35:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 26 21:34:50.590454602 (57h59m0.313017632s) [upload=55654]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:34:50.590461004 (57h59m0.313024044s) [upload=55654]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:34:51.058371196 (57h59m0.780934246s) [seg_pos={0 0}, upload=55654]: Begin upload segment... Jul 26 21:34:51.058434267 (57h59m0.780997307s) [seg_pos={0 0}, upload=55654]: Done begin upload segment. Jul 26 21:34:51.058457972 (57h59m0.781021002s) [noise=true , address=104.36.82.53:4153 , piece_id=JSITHZQHE7RIDILNYJHJ7MZ7MUWZ36BBIR3BAELBI4VCFEFQ5RPA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.058846295 (57h59m0.781409335s) [noise=true , address=104.36.82.13:4207 , piece_id=U527R2G4KOSDSHMWEA6ESAOPEURIWWZJKMXCZWFPEPHGTQYIV5DA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.058864099 (57h59m0.781427139s) [noise=true , address=104.36.82.7:4931 , piece_id=2XRG33HE2DQPM5KL3AW62TH6RCG2YJXUZDBO4ZEFKNPJ33TEGUSA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.059123064 (57h59m0.781686104s) [noise=true , address=104.36.82.22:4543 , piece_id=IGJC6PUCA64L5TIRPJV3CV7IL6JBSHSHRQR6HZCL6JKMOLYVXDEA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.059467123 (57h59m0.782030163s) [noise=true , address=104.36.82.43:4836 , piece_id=3C4X3INXPIU6PA24UENPXCYO6YPBLPO4UPF36KKGERHNQWQY2FWA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.059603033 (57h59m0.782166073s) [noise=true , address=104.36.82.52:4117 , piece_id=C4ARISN33TG2Z75ZM4P7YT6JGDT4TGQKBS7UD3V5WZ7YMAUWQFTA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.059763761 (57h59m0.782326811s) [noise=true , address=104.36.82.50:4034 , piece_id=EI2BZRKUT56RQSQ2SGJUQSL5U7Z5L6YLE3HRCAEDIO26IFVB27DQ, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.059853933 (57h59m0.782416963s) [noise=true , address=104.36.82.8:4994 , piece_id=JAHDAXDGSIS7FR64MT7MDSBNLCT4DWL5DC2TATNVPG56JMCDINDA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060046662 (57h59m0.782609702s) [noise=true , address=104.36.82.54:4191 , piece_id=TXBLNZKAHWKERTGXU6HAD4INJ6N6XZE2SITWBN2L2SG42SC6X6SA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060085175 (57h59m0.782648215s) [noise=true , address=104.36.82.54:4182 , piece_id=AM4TO7G5NVMEV2VBE5ZQZVPVCVU3PIBLG5WDUMC7LLTKL7MBAIYQ, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060324063 (57h59m0.782887103s) [noise=true , address=79.127.205.233:4647 , piece_id=GOKF2BSQ4VBFD7WTL3QED3277S74X2CMWMEU3OMABNHIJ3ZDIX2Q, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060343139 (57h59m0.782906169s) [noise=true , address=104.36.82.47:4518 , piece_id=JEC6O5EN2I453FWWGU43EE4YK5KWCXHVAMGFEMVP42JUDDSYLV3A, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060561427 (57h59m0.783124467s) [noise=true , address=104.36.82.5:4860 , piece_id=ISB6O6IVKHXLQ5TS734TBJUE2KR36F3ZORCPSOROKA5PAW5E7C6Q, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060605110 (57h59m0.78316815s) [noise=true , address=64.49.13.170:4721 , piece_id=IDQP2IKOKJIDDFQ7LVOIBXNDQM4DBRWTJX5ZLZM5ZCMOVGCZMMRA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.060853125 (57h59m0.783416165s) [noise=true , address=104.36.82.42:4803 , piece_id=46NOG27NWNOO7USQZZLSQKRXSHS6ZZDYFJ4G67PNMZPDCRRHY7KA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.061156415 (57h59m0.783719455s) [noise=true , address=79.127.205.245:4220 , piece_id=YFX3EHNBLM7YFT4TPIXLBIPZDZPMBJPAGDMF6SEDWHFTLZM7X7PA, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.061418717 (57h59m0.783981757s) [noise=true , address=104.36.82.7:4923 , piece_id=B5IYIUUOXFHDQL2YA2QHIHLPJQI66ZSTH6527MMU4UD32JMHN63Q, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:51.061662023 (57h59m0.784225063s) [noise=true , address=104.36.82.55:4194 , piece_id=OTHEUQDH7IEM5J757GAY2M43OIMOYOSWNG2CCFMM3UHGE7GGO3AQ, seg_pos={0 0}, upload=55654]: Uploading piece... Jul 26 21:34:52.159508112 (57h59m1.882071153s) [upload=55654]: Next returned nil segment Jul 26 21:34:52.159525926 (57h59m1.882088966s) [upload=55654]: Waiting for error group managing segments... Jul 26 21:34:52.594017768 (57h59m2.316580808s) [noise=true , address=79.127.205.245:4220 , piece_id=YFX3EHNBLM7YFT4TPIXLBIPZDZPMBJPAGDMF6SEDWHFTLZM7X7PA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:34:52.620314943 (57h59m2.342878013s) [noise=true , address=79.127.205.233:4647 , piece_id=GOKF2BSQ4VBFD7WTL3QED3277S74X2CMWMEU3OMABNHIJ3ZDIX2Q, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:34:52.761899450 (57h59m2.48446248s) [noise=true , address=64.49.13.170:4721 , piece_id=IDQP2IKOKJIDDFQ7LVOIBXNDQM4DBRWTJX5ZLZM5ZCMOVGCZMMRA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:34:53.102602534 (57h59m2.825165574s) [noise=true , address=104.36.82.22:4543 , piece_id=IGJC6PUCA64L5TIRPJV3CV7IL6JBSHSHRQR6HZCL6JKMOLYVXDEA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:02.587595211 (57h59m12.310158251s) [noise=true , address=104.36.82.7:4923 , piece_id=B5IYIUUOXFHDQL2YA2QHIHLPJQI66ZSTH6527MMU4UD32JMHN63Q, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:02.624092011 (57h59m12.346655051s) [noise=true , address=104.36.82.7:4931 , piece_id=2XRG33HE2DQPM5KL3AW62TH6RCG2YJXUZDBO4ZEFKNPJ33TEGUSA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:12.638826116 (57h59m22.361389156s) [noise=true , address=104.36.82.43:4836 , piece_id=3C4X3INXPIU6PA24UENPXCYO6YPBLPO4UPF36KKGERHNQWQY2FWA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:12.820125419 (57h59m22.542688459s) [noise=true , address=104.36.82.5:4860 , piece_id=ISB6O6IVKHXLQ5TS734TBJUE2KR36F3ZORCPSOROKA5PAW5E7C6Q, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:15.936272940 (57h59m25.65883598s) [noise=true , address=104.36.82.42:4803 , piece_id=46NOG27NWNOO7USQZZLSQKRXSHS6ZZDYFJ4G67PNMZPDCRRHY7KA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:16.449041001 (57h59m26.171604031s) [noise=true , address=104.36.82.8:4994 , piece_id=JAHDAXDGSIS7FR64MT7MDSBNLCT4DWL5DC2TATNVPG56JMCDINDA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:19.050564337 (57h59m28.773127377s) [noise=true , address=104.36.82.13:4207 , piece_id=U527R2G4KOSDSHMWEA6ESAOPEURIWWZJKMXCZWFPEPHGTQYIV5DA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:21.262985040 (57h59m30.98554808s) [noise=true , address=104.36.82.50:4034 , piece_id=EI2BZRKUT56RQSQ2SGJUQSL5U7Z5L6YLE3HRCAEDIO26IFVB27DQ, seg_pos={0 0}, upload=55654]: Done uploading piece. err: <nil> Jul 26 21:35:49.947576995 (57h59m59.670140035s) [noise=true , address=104.36.82.54:4191 , piece_id=TXBLNZKAHWKERTGXU6HAD4INJ6N6XZE2SITWBN2L2SG42SC6X6SA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: ecclient: upload failed (node:12d7ChCRcN4pJjJ51aSAPitg3zTRxr5eVbRXVZcvbi6t1gifWmw, address:104.36.82.54:4191): protocol: rpc: dial tcp 104.36.82.54:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:49.947623304 (57h59m59.670186344s) [noise=true , address=104.36.82.54:4182 , piece_id=AM4TO7G5NVMEV2VBE5ZQZVPVCVU3PIBLG5WDUMC7LLTKL7MBAIYQ, seg_pos={0 0}, upload=55654]: Done uploading piece. err: ecclient: upload failed (node:12HbkH31sfDUnZLXAbmzQphcFYmzB8MQo98Q6Ufbo59et42zxMy, address:104.36.82.54:4182): protocol: rpc: dial tcp 104.36.82.54:4182: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:49.947656868 (57h59m59.670219908s) [noise=true , address=104.36.82.47:4518 , piece_id=JEC6O5EN2I453FWWGU43EE4YK5KWCXHVAMGFEMVP42JUDDSYLV3A, seg_pos={0 0}, upload=55654]: Done uploading piece. err: ecclient: upload failed (node:17TZ9JLdCivFF46guhSTW6hcoUmTz5uo5wvLkDqC4BGmpm9eLb, address:104.36.82.47:4518): protocol: rpc: dial tcp 104.36.82.47:4518: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:49.947697456 (57h59m59.670260496s) [noise=true , address=104.36.82.53:4153 , piece_id=JSITHZQHE7RIDILNYJHJ7MZ7MUWZ36BBIR3BAELBI4VCFEFQ5RPA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: ecclient: upload failed (node:1fPp4DvLe18dZWPmMPFiC8o9jL4uiEFNmPAHHmFEH74ZXNndw3, address:104.36.82.53:4153): protocol: rpc: dial tcp 104.36.82.53:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:49.947726601 (57h59m59.670289641s) [noise=true , address=104.36.82.52:4117 , piece_id=C4ARISN33TG2Z75ZM4P7YT6JGDT4TGQKBS7UD3V5WZ7YMAUWQFTA, seg_pos={0 0}, upload=55654]: Done uploading piece. err: ecclient: upload failed (node:1QVEhFpjfLubGsZmoY1XQdDWevLy2xWdjtPq4JnbSYuBgEXgz7, address:104.36.82.52:4117): protocol: rpc: dial tcp 104.36.82.52:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:49.947764433 (57h59m59.670327473s) [noise=true , address=104.36.82.55:4194 , piece_id=OTHEUQDH7IEM5J757GAY2M43OIMOYOSWNG2CCFMM3UHGE7GGO3AQ, seg_pos={0 0}, upload=55654]: Done uploading piece. err: ecclient: upload failed (node:12kta1F6RNDP9GJdkiUG3x1Nk6juWJ2tmSDr599DYomSq9C1BBh, address:104.36.82.55:4194): protocol: write tcp 188.166.225.170:55268->104.36.82.55:4194: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:35:49.947779423 (57h59m59.670342463s) [seg_pos={0 0}, upload=55654]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 21:35:50.177703090 (57h59m59.900266131s) [upload=55654]: Done uploading segments... Jul 26 21:35:50.177721124 (57h59m59.900284164s) [upload=55654]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.231865747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X....X....] duration: 8.339426369s FAIL (2025-07-26 21:35:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.339454002s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.........X.................X] duration: 1.653247134s FAIL (2025-07-26 21:35:20 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.653278324s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X...................................XX..] duration: 12.361309026s FAIL (2025-07-26 21:35:12 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 12.361335055s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X....X...] duration: 10.824114523s FAIL (2025-07-26 21:35:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 10.824142868s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 1m0.233402221s FAIL (2025-07-26 21:34:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== eg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.795019865 (57h57m53.517582905s) [noise=true , address=79.127.205.246:4279 , piece_id=N66CVDXFAZRL2JS4BOZZWKEUIXKCBTCLKFX4TD4HJUWAOYBCOD5A, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.795293749 (57h57m53.517856789s) [noise=true , address=104.36.82.20:4440 , piece_id=TGFBCPQHGUOLSLZ6NPX27RXX4ICLLIRQUCDXEUEZIIEMZDSJ7P2Q, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.795540020 (57h57m53.51810306s) [noise=true , address=79.127.205.245:4227 , piece_id=2E7WJ6G3ABG7HFN6CM6NEMHKQXV6AEATTFEG2K3TLQOUV4VEH3UA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.795602259 (57h57m53.518165339s) [noise=true , address=104.36.82.53:4139 , piece_id=BFXZ4DA6LT5YHTDRFDZUBXNJ44WZKYZBBKEQIDGMB7DAPSUEM6YA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.795787444 (57h57m53.518350484s) [noise=true , address=64.49.13.140:4514 , piece_id=RIYBZWYEV7AP65OXVQHJQNR3NWTA2GXFWCCD7C3MOTHJZ6LDUEAA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796030629 (57h57m53.518593669s) [noise=true , address=104.36.82.56:4258 , piece_id=JUDTN66VDLLBWQVOSSC5OHXAAMPUJG6U5ILUTIGPSQ7CVC7MIW5A, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796116954 (57h57m53.518679994s) [noise=true , address=77.90.31.17:4774 , piece_id=L5FOGPIGHI4YYRWHAEG54KUVPVCBXJI7UWB6WQUPFKUFP2Q2VSLQ, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796293521 (57h57m53.518856562s) [noise=true , address=104.36.82.54:4176 , piece_id=444SRJYF2DX2IYWIFOQBZD74MVZ5ZN5Z35BWV3KJAZSAVQ65BIXA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796474849 (57h57m53.519037889s) [noise=true , address=104.36.82.46:4499 , piece_id=73AB6ODF4VKVFBBFOD56KCXXDVW6WH77KVCQJXJTIQDBMG5E42HA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796515867 (57h57m53.519078907s) [noise=true , address=104.36.82.55:4198 , piece_id=VO7NW3XQ2QSCZ2GLLEAXQ2MMNGTMXXJSWL2BNV7MDZJWCS3SY34A, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796726360 (57h57m53.51928939s) [noise=true , address=104.36.82.47:4531 , piece_id=AOEQB3WUJA34NBFXBJK6ZC73O2JJ54OMILTU2YSZ6DONV3YJJCLQ, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796755235 (57h57m53.519318265s) [noise=true , address=104.36.82.12:4161 , piece_id=RQWYOT6G3VNJEJNMFCAYIPWDK6WUDR4FAEHIJNX5HVBPEC4SEHRA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.796946291 (57h57m53.519509331s) [noise=true , address=79.127.201.215:4354 , piece_id=H4EEZLT43VM7YQ5ZUEUUM5JF6WAGV22SFWOCEVPJWBTONIQSJXCA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.797170149 (57h57m53.519733179s) [noise=true , address=104.36.82.51:4067 , piece_id=NCNCERDE4MD2ELMPEQF6ZPUV3E4VEWXXB3KCDH6LLVFUQPKPHDYQ, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.797364110 (57h57m53.51992715s) [noise=true , address=77.90.31.13:4644 , piece_id=OIF2WYFAHOW2YRRAF7L3NAYXWGH7OTX3AMY2LK42UAUXKDDJX6FA, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.797557059 (57h57m53.520120099s) [noise=true , address=79.127.205.235:4666 , piece_id=E3DBFFL2V5JX554BRYSKD2QJSJJ6JRTGD7ZW4CYV4WXV3AFHNVMQ, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:43.798378351 (57h57m53.520941391s) [noise=true , address=104.36.82.3:4799 , piece_id=Z5TRMRLI44LBQNRFRYB2GTMY5KFJ3JHWV5NFZ2ARJIE4ESPMLX5A, seg_pos={0 0}, upload=55635]: Uploading piece... Jul 26 21:33:44.891574555 (57h57m54.614137595s) [upload=55635]: Next returned nil segment Jul 26 21:33:44.891611757 (57h57m54.614174797s) [upload=55635]: Waiting for error group managing segments... Jul 26 21:33:45.177901310 (57h57m54.90046435s) [noise=true , address=79.127.205.235:4666 , piece_id=E3DBFFL2V5JX554BRYSKD2QJSJJ6JRTGD7ZW4CYV4WXV3AFHNVMQ, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:45.210257318 (57h57m54.932820358s) [noise=true , address=79.127.205.246:4279 , piece_id=N66CVDXFAZRL2JS4BOZZWKEUIXKCBTCLKFX4TD4HJUWAOYBCOD5A, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:45.444414285 (57h57m55.166977375s) [noise=true , address=79.127.205.245:4227 , piece_id=2E7WJ6G3ABG7HFN6CM6NEMHKQXV6AEATTFEG2K3TLQOUV4VEH3UA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:45.903941069 (57h57m55.626504109s) [noise=true , address=104.36.82.16:4325 , piece_id=KOZXZM37VKS2L7JDW3YE5JYUAT3RWRYOMZNHJI3LUAKY5RGI4UNA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:46.120547717 (57h57m55.843110747s) [noise=true , address=64.49.13.140:4514 , piece_id=RIYBZWYEV7AP65OXVQHJQNR3NWTA2GXFWCCD7C3MOTHJZ6LDUEAA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:47.013084504 (57h57m56.735647534s) [noise=true , address=79.127.201.215:4354 , piece_id=H4EEZLT43VM7YQ5ZUEUUM5JF6WAGV22SFWOCEVPJWBTONIQSJXCA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:47.899039880 (57h57m57.62160292s) [noise=true , address=77.90.31.17:4774 , piece_id=L5FOGPIGHI4YYRWHAEG54KUVPVCBXJI7UWB6WQUPFKUFP2Q2VSLQ, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:33:47.985693556 (57h57m57.708256596s) [noise=true , address=77.90.31.13:4644 , piece_id=OIF2WYFAHOW2YRRAF7L3NAYXWGH7OTX3AMY2LK42UAUXKDDJX6FA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:34:21.614936612 (57h58m31.337499652s) [noise=true , address=104.36.82.51:4067 , piece_id=NCNCERDE4MD2ELMPEQF6ZPUV3E4VEWXXB3KCDH6LLVFUQPKPHDYQ, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:34:28.865644037 (57h58m38.588207067s) [noise=true , address=104.36.82.3:4799 , piece_id=Z5TRMRLI44LBQNRFRYB2GTMY5KFJ3JHWV5NFZ2ARJIE4ESPMLX5A, seg_pos={0 0}, upload=55635]: Done uploading piece. err: <nil> Jul 26 21:34:42.709374324 (57h58m52.431937374s) [noise=true , address=104.36.82.56:4258 , piece_id=JUDTN66VDLLBWQVOSSC5OHXAAMPUJG6U5ILUTIGPSQ7CVC7MIW5A, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:12g7DdMjQHyYTD9hRvixfNedxyPr6q6uE2rLdEbhQ1MGANRMDm3, address:104.36.82.56:4258): protocol: rpc: dial tcp 104.36.82.56:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709427716 (57h58m52.431990756s) [noise=true , address=104.36.82.54:4176 , piece_id=444SRJYF2DX2IYWIFOQBZD74MVZ5ZN5Z35BWV3KJAZSAVQ65BIXA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:1Yi5AibNiCNV8vUb9MKgooHCw8Nr79m9HqTrkrYybNZmatg9PD, address:104.36.82.54:4176): protocol: rpc: dial tcp 104.36.82.54:4176: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709469356 (57h58m52.432032396s) [noise=true , address=104.36.82.55:4198 , piece_id=VO7NW3XQ2QSCZ2GLLEAXQ2MMNGTMXXJSWL2BNV7MDZJWCS3SY34A, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:104.36.82.55:4198): protocol: write tcp 188.166.225.170:56108->104.36.82.55:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709499734 (57h58m52.432062784s) [noise=true , address=104.36.82.20:4440 , piece_id=TGFBCPQHGUOLSLZ6NPX27RXX4ICLLIRQUCDXEUEZIIEMZDSJ7P2Q, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:12GfuKp6kJEy1NTHpo9i21UDm62hrAtKq7124Z6FUehNyyLQ33i, address:104.36.82.20:4440): protocol: rpc: dial tcp 104.36.82.20:4440: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709558066 (57h58m52.432121106s) [noise=true , address=104.36.82.53:4139 , piece_id=BFXZ4DA6LT5YHTDRFDZUBXNJ44WZKYZBBKEQIDGMB7DAPSUEM6YA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:12T99BLYzWYXV5VkEKAYQVyzoeHA4XYz4Jrh8Dz3K6jqkec52Lg, address:104.36.82.53:4139): protocol: write tcp 188.166.225.170:50470->104.36.82.53:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709593874 (57h58m52.432156904s) [noise=true , address=104.36.82.46:4499 , piece_id=73AB6ODF4VKVFBBFOD56KCXXDVW6WH77KVCQJXJTIQDBMG5E42HA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: write tcp 188.166.225.170:37780->104.36.82.46:4499: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709628380 (57h58m52.43219142s) [noise=true , address=104.36.82.47:4531 , piece_id=AOEQB3WUJA34NBFXBJK6ZC73O2JJ54OMILTU2YSZ6DONV3YJJCLQ, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:12LZprfyqfD2YUwgAarzyjbgHEb6jMhAMAday5C45xk3j9RUS4u, address:104.36.82.47:4531): protocol: rpc: dial tcp 104.36.82.47:4531: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709635794 (57h58m52.432198835s) [noise=true , address=104.36.82.12:4161 , piece_id=RQWYOT6G3VNJEJNMFCAYIPWDK6WUDR4FAEHIJNX5HVBPEC4SEHRA, seg_pos={0 0}, upload=55635]: Done uploading piece. err: ecclient: upload failed (node:1SdSwg6tZCKSturNytQmWUmSWQ2G5dUSrb2RA8hxLLpBtBE6tf, address:104.36.82.12:4161): protocol: write tcp 188.166.225.170:37560->104.36.82.12:4161: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:34:42.709667535 (57h58m52.432230575s) [seg_pos={0 0}, upload=55635]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 26 21:34:42.941179192 (57h58m52.663742222s) [upload=55635]: Done uploading segments... Jul 26 21:34:42.941204590 (57h58m52.66376762s) [upload=55635]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.233414926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X.........X.................] duration: 1.856701093s FAIL (2025-07-26 21:34:38 UTC) orbiter warning: uplink: too many requests Duration: 1.856729007s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X....X..] duration: 7.59125504s FAIL (2025-07-26 21:34:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.59131245s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X...................................XX.] duration: 10.488931727s FAIL (2025-07-26 21:34:12 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 10.488950021s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.....................X...................................XX] duration: 30.241749447s FAIL (2025-07-26 21:34:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.241772202s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.........X................] duration: 1.875077633s FAIL (2025-07-26 21:33:53 UTC) orbiter warning: uplink: too many requests Duration: 1.875110085s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 1m0.232572859s FAIL (2025-07-26 21:33:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== load=55618]: Uploading piece... Jul 26 21:32:43.338615403 (57h56m53.061178443s) [noise=true , address=104.36.82.38:4722 , piece_id=N72D4CF475YEMNDWFAJS5YAPD5SWGEWF4CEAEKHDWXHEE3RGNZTQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.339132072 (57h56m53.061695132s) [noise=true , address=104.36.82.46:4490 , piece_id=3PEEMNTLGH2FO6TOC77QORPXXW35O2WBX444LFSEVXYLIYL53OZA, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.339557255 (57h56m53.062120315s) [noise=true , address=79.127.205.245:4238 , piece_id=OFUAAII5MMJNRGBMTN7ZHHG5VVBIAZGBG54AJNRGDRFBOAWJMBGA, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.339936431 (57h56m53.062499461s) [noise=true , address=79.127.201.215:4349 , piece_id=EQNA3VQD6SFEDACKDXHTISEFAQNFB22K2YULE4ML6FJB2WKEEXYQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.340320746 (57h56m53.062883796s) [noise=true , address=104.36.82.9:4014 , piece_id=2HVTEVHW2RF3CODDKONIQMIVLAELZ4NWLJNQYOYOSDXVCPYT7UBQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.340654294 (57h56m53.063217334s) [noise=true , address=104.36.82.47:4521 , piece_id=XM5MWKKEC7GFNDMGL4ENM2XCZA7UMCECG2PGDCTLZSXAT5MGJIXQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.340984766 (57h56m53.063547806s) [noise=true , address=104.36.82.54:4171 , piece_id=QNJDF6T6HCM3RNVLN7AASCTUYKIDLGKHJPOCVDQDY4YR5JKVOXLQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.341390523 (57h56m53.063953563s) [noise=true , address=77.90.31.18:4831 , piece_id=3R64HTKKN5OIL4VJ4CZMNFJQ26UTTVOZYRE7NZTPOYDICNLQF2BQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.341767264 (57h56m53.064330304s) [noise=true , address=104.36.82.52:4098 , piece_id=RBY6I6FR6XWNUNUNOFKOPBKMJQFNLRXWQAKX2BAVK25YRXCNFJNQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.342101153 (57h56m53.064664193s) [noise=true , address=77.90.31.13:4658 , piece_id=FPNACYUDUCHZ5GXMJWLYY6ZNGVILA5HIECWBGGLCQFWTXCVVK64Q, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.342395486 (57h56m53.064958526s) [noise=true , address=64.49.13.155:4923 , piece_id=CRVRM7U27IVA4XNIT4WNVLTWBDV336B3LEWUY236T3QE5GYCSGPA, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.342725047 (57h56m53.065288087s) [noise=true , address=79.127.205.235:4678 , piece_id=JRTZ6OBPIGRSFGKXWYWHPUKPWMJ3YKDJB3V23R26JL5VMGJFDOPA, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.343039919 (57h56m53.065602959s) [noise=true , address=79.127.219.38:4018 , piece_id=GDONHRAVZDMUBGDWTSH6CFQEDFT5CP7YZSD3TRMZSIXEFGFLIFMA, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.343342918 (57h56m53.065905968s) [noise=true , address=104.36.82.53:4122 , piece_id=3Z63ATSWNFD5N7OF42NKJMR4TSULHKDZIGN4GZOHRY6QFUADSSEQ, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:43.343643844 (57h56m53.066206884s) [noise=true , address=77.90.31.21:4897 , piece_id=AMPEQSS3NLWG5FCWMYFUHABG2YG6CORGFAAFFEJXB3MTJYAPS7KA, seg_pos={0 0}, upload=55618]: Uploading piece... Jul 26 21:32:44.435847500 (57h56m54.15841054s) [upload=55618]: Next returned nil segment Jul 26 21:32:44.435867930 (57h56m54.15843096s) [upload=55618]: Waiting for error group managing segments... Jul 26 21:32:44.733161578 (57h56m54.455724618s) [noise=true , address=79.127.205.245:4238 , piece_id=OFUAAII5MMJNRGBMTN7ZHHG5VVBIAZGBG54AJNRGDRFBOAWJMBGA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:44.809863453 (57h56m54.532426493s) [noise=true , address=79.127.205.235:4678 , piece_id=JRTZ6OBPIGRSFGKXWYWHPUKPWMJ3YKDJB3V23R26JL5VMGJFDOPA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:44.810001767 (57h56m54.532564807s) [noise=true , address=64.49.13.155:4923 , piece_id=CRVRM7U27IVA4XNIT4WNVLTWBDV336B3LEWUY236T3QE5GYCSGPA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:45.222912748 (57h56m54.945475778s) [noise=true , address=45.91.116.10:4100 , piece_id=ZOK2ZRQN5ONQD7VGWERSXUXUFWUG2IBO2YDNSPEYGHUGZJ54IWKA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:45.305709455 (57h56m55.028272525s) [noise=true , address=79.127.219.38:4018 , piece_id=GDONHRAVZDMUBGDWTSH6CFQEDFT5CP7YZSD3TRMZSIXEFGFLIFMA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:45.694739895 (57h56m55.417302935s) [noise=true , address=79.127.201.215:4349 , piece_id=EQNA3VQD6SFEDACKDXHTISEFAQNFB22K2YULE4ML6FJB2WKEEXYQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:48.232458244 (57h56m57.955021284s) [noise=true , address=77.90.31.21:4897 , piece_id=AMPEQSS3NLWG5FCWMYFUHABG2YG6CORGFAAFFEJXB3MTJYAPS7KA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:48.529619037 (57h56m58.252182067s) [noise=true , address=77.90.31.13:4658 , piece_id=FPNACYUDUCHZ5GXMJWLYY6ZNGVILA5HIECWBGGLCQFWTXCVVK64Q, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:32:49.283263151 (57h56m59.005826191s) [noise=true , address=77.90.31.18:4831 , piece_id=3R64HTKKN5OIL4VJ4CZMNFJQ26UTTVOZYRE7NZTPOYDICNLQF2BQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: <nil> Jul 26 21:33:42.244828443 (57h57m51.967391483s) [noise=true , address=104.36.82.47:4521 , piece_id=XM5MWKKEC7GFNDMGL4ENM2XCZA7UMCECG2PGDCTLZSXAT5MGJIXQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD, address:104.36.82.47:4521): protocol: write tcp 188.166.225.170:54844->104.36.82.47:4521: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.244877646 (57h57m51.967440677s) [noise=true , address=104.36.82.21:4478 , piece_id=UYYZWG6XU5OXSGJAMYX6YN2LCKCJ5NNOP7ISDCDB7ROGYRZ5H5VQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:17kFjjJyvHApPUxUHJ6HeMEBCoo6xTbr8QizXjBEHwWj3mZUha, address:104.36.82.21:4478): protocol: write tcp 188.166.225.170:45706->104.36.82.21:4478: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.244908206 (57h57m51.967471246s) [noise=true , address=104.36.82.55:4210 , piece_id=4BK53GIYE5EYB7AK67GDXTCTTOMHLKQRAC6D2HH6KW22EDNK2PRQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:12KuoSWvJCc3mfowPPSJkdCDewkrHZnG6gz1fWcmJRGR5wvHzLL, address:104.36.82.55:4210): protocol: write tcp 188.166.225.170:33192->104.36.82.55:4210: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245023126 (57h57m51.967586166s) [noise=true , address=104.36.82.53:4122 , piece_id=3Z63ATSWNFD5N7OF42NKJMR4TSULHKDZIGN4GZOHRY6QFUADSSEQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:124d52DhtbtwipVpZuWdpFTQAttR18hy1K1BbokNRX4R1bLtzTi, address:104.36.82.53:4122): protocol: write tcp 188.166.225.170:41042->104.36.82.53:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245066037 (57h57m51.967629077s) [noise=true , address=104.36.82.52:4098 , piece_id=RBY6I6FR6XWNUNUNOFKOPBKMJQFNLRXWQAKX2BAVK25YRXCNFJNQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:1SoFNdwfh4p5tXUkpfjQaev6tdA2ZdhsXUFvZNBsy5QrJ9zdqb, address:104.36.82.52:4098): protocol: write tcp 188.166.225.170:44574->104.36.82.52:4098: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245089682 (57h57m51.967652723s) [noise=true , address=104.36.82.46:4490 , piece_id=3PEEMNTLGH2FO6TOC77QORPXXW35O2WBX444LFSEVXYLIYL53OZA, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:104.36.82.46:4490): protocol: write tcp 188.166.225.170:56976->104.36.82.46:4490: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245142844 (57h57m51.967705874s) [noise=true , address=104.36.82.38:4722 , piece_id=N72D4CF475YEMNDWFAJS5YAPD5SWGEWF4CEAEKHDWXHEE3RGNZTQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:12Zw3dtUg9gARz52KhwDLTjcqC3CNm7dpPzpdx7P7PbL1jwe8uc, address:104.36.82.38:4722): protocol: write tcp 188.166.225.170:39174->104.36.82.38:4722: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245153244 (57h57m51.967716284s) [noise=true , address=104.36.82.54:4171 , piece_id=QNJDF6T6HCM3RNVLN7AASCTUYKIDLGKHJPOCVDQDY4YR5JKVOXLQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:125SMhDsEZruNu6LkmQn2M7R6WzvXTdk86DRsKf522u8BNiANq3, address:104.36.82.54:4171): protocol: write tcp 188.166.225.170:35668->104.36.82.54:4171: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245157883 (57h57m51.967720944s) [noise=true , address=104.36.82.9:4014 , piece_id=2HVTEVHW2RF3CODDKONIQMIVLAELZ4NWLJNQYOYOSDXVCPYT7UBQ, seg_pos={0 0}, upload=55618]: Done uploading piece. err: ecclient: upload failed (node:12uK9ULUdABDm5XBP4nh1omWhR5qoQerFsCmL4bQ9dmNespXpTy, address:104.36.82.9:4014): protocol: write tcp 188.166.225.170:60916->104.36.82.9:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:42.245176989 (57h57m51.967740029s) [seg_pos={0 0}, upload=55618]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 26 21:33:42.475923591 (57h57m52.198486621s) [upload=55618]: Done uploading segments... Jul 26 21:33:42.475951805 (57h57m52.198514835s) [upload=55618]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.232584441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration: 9.613470375s FAIL (2025-07-26 21:33:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.613486917s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X....X] duration: 30.232925847s FAIL (2025-07-26 21:33:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== , address=104.36.82.13:4186 , piece_id=B66ELIOOQMCFCM5FZ5OYTJX5YJRGI3SUICIYOCNDGCP3FZ6R7ONQ, seg_pos={0 0}, upload=55611]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:45112->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:23.846860758 (57h56m33.569423838s) [seg_pos={0 0}, upload=55611]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:32:23.846960069 (57h56m33.569523109s) [upload=55611]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:32:23.846967833 (57h56m33.569530923s) [upload=55611]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:32:24.140008393 (57h56m33.862571433s) [upload=55611]: Done uploading segments... Jul 26 21:32:24.140030354 (57h56m33.862593394s) [upload=55611]: Upload finished. err: <nil> Jul 26 21:32:58.848628916 (57h57m8.571191956s) [upload=55625]: Starting upload Jul 26 21:32:58.848693470 (57h57m8.57125651s) [upload=55625]: Done starting upload Jul 26 21:32:58.848804993 (57h57m8.571368033s) [upload=55625]: Uploading segments... Jul 26 21:32:58.849100608 (57h57m8.571663648s) [upload=55625]: Got next segment. Inline: false Jul 26 21:32:58.849162127 (57h57m8.571725167s) [upload=55625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:32:58.849169350 (57h57m8.57173239s) [upload=55625]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:32:59.081501507 (57h57m8.804064547s) [seg_pos={0 0}, upload=55625]: Begin upload segment... Jul 26 21:32:59.081543577 (57h57m8.804106617s) [seg_pos={0 0}, upload=55625]: Done begin upload segment. Jul 26 21:32:59.081560720 (57h57m8.80412376s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081568745 (57h57m8.804131785s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081608090 (57h57m8.80417113s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081938682 (57h57m8.804501762s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.081949754 (57h57m8.804512794s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082209921 (57h57m8.804772961s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082260598 (57h57m8.804823638s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082518673 (57h57m8.805081713s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.082773390 (57h57m8.80533642s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083059387 (57h57m8.805622427s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083285820 (57h57m8.80584886s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083495431 (57h57m8.806058471s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083705594 (57h57m8.806268634s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.083901268 (57h57m8.806464298s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.085955498 (57h57m8.808518538s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:32:59.087265113 (57h57m8.809828154s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Uploading piece... Jul 26 21:33:00.770004301 (57h57m10.492567331s) [upload=55625]: Next returned nil segment Jul 26 21:33:00.770022166 (57h57m10.492585206s) [upload=55625]: Waiting for error group managing segments... Jul 26 21:33:01.103325942 (57h57m10.825888982s) [noise=true , address=45.91.116.5:4298 , piece_id=7CGF6VDLSHLZH4LME6SNU3G55HVCW4H7RT2EE5C77MNTGWKBFG5A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.103780683 (57h57m10.826343743s) [noise=true , address=45.91.116.5:4299 , piece_id=Q25W2W25GM7BOCIE5JWXXX3EN6QG6QCADRJ3NBMMA2GY6ERSQH7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.122250531 (57h57m10.844813581s) [noise=true , address=45.91.116.7:4011 , piece_id=ATQ6DW4LQ3UUK6RHOX6RZ7N3TAAD33NDL4SAU6H36WERV4KOG54A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.185003338 (57h57m10.907566378s) [noise=true , address=45.91.116.13:4154 , piece_id=2XAOLRQV457NXAPEJPYLFLHOSLUYREP2ZFPO73QZHLLNDRWRXW7Q, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.192093133 (57h57m10.914656173s) [noise=true , address=45.91.116.9:4261 , piece_id=LIKCH6ZJNMIB5RWJENQMHLOM6MXKAABG4M556JRG2Y4HOOOV5JYA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.205502219 (57h57m10.928065259s) [noise=true , address=45.91.116.12:4226 , piece_id=WYYIGJZBOPCG6KDROLZXHYOWKUBQVLDMN554V32QI7AWCE7JPKMQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.206594729 (57h57m10.929157769s) [noise=true , address=45.91.116.11:4118 , piece_id=HMBWSCR4FHW5RFCMJ3KY7SX3TXT52B7BKNCOP2IQ3D5FNIJSJRYQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.252397474 (57h57m10.974960514s) [noise=true , address=45.91.116.10:4082 , piece_id=2E4Y7CEJEY4D7FWGFTP2YTFIH2Q2GE3YNBQUXILN44ZOEIDUDRPQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.257171729 (57h57m10.979734769s) [noise=true , address=45.91.116.7:4009 , piece_id=E6LLVKKKLBNTC4P75MP747B4TII7VAFYDCYU5UDWPEEUTVUZDN3A, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:01.275092778 (57h57m10.997655818s) [noise=true , address=45.91.116.6:4975 , piece_id=3QW3RWSSZHRYPP7SSVE7CJN57GVFCFQIRNM3SUS2IPEMNEWTBNCQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:04.577165881 (57h57m14.299728921s) [noise=true , address=77.90.31.17:4772 , piece_id=GQFJ672UHU3N4EV322G5K57RRNBTEK6YZ7JEAPAPC4K3JUEQTDFA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:05.192932570 (57h57m14.91549562s) [noise=true , address=77.90.31.18:4807 , piece_id=EZRQAN27S4MPXSSTL4UC6MGT5UZXA4D3RAQPAF34RZ3WB7EM3TTA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: <nil> Jul 26 21:33:28.205828302 (57h57m37.928391342s) [noise=true , address=104.36.82.11:4115 , piece_id=TBMHH56ITXQ7QM3L36B4KMOTKJSGM2JHXURCPLDVERYHT2WS4MRQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205917181 (57h57m37.928480282s) [noise=true , address=104.36.82.11:4114 , piece_id=CBI3XXQ4SDZIIRYEM4E7PHF3PQIKANOXPOYUYIKTRJRIK5WZZ5YQ, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: rpc: dial tcp 104.36.82.11:4114: i/o timeout; conn closed: op Jul 26 21:33:28.205981074 (57h57m37.928544114s) [noise=true , address=104.36.82.6:4874 , piece_id=FBO2BWW5XLUYQLL6F4MNKTARGSQFIPLLE2N6O6M3LQLRIT3XYEWA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 188.166.225.170:43608->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205985783 (57h57m37.928548823s) [noise=true , address=104.36.82.2:4730 , piece_id=X76NV6YWUHUGKF2E6VMW353J2H3SI2O7E7XIM2V3SY6FHR5QYIIA, seg_pos={0 0}, upload=55625]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:33:28.205999188 (57h57m37.928562228s) [seg_pos={0 0}, upload=55625]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:33:28.437000046 (57h57m38.159563086s) [upload=55625]: Done uploading segments... Jul 26 21:33:28.437032148 (57h57m38.159595208s) [upload=55625]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.232952189s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.....................X...................................X] duration: 30.232553631s FAIL (2025-07-26 21:33:19 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.232576225s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.238708657s FAIL (2025-07-26 21:32:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== pb.BatchRequestItem_SegmentBegin Jul 26 21:31:42.877847417 (57h55m52.600410447s) [seg_pos={0 0}, upload=55603]: Begin upload segment... Jul 26 21:31:42.877880420 (57h55m52.60044346s) [seg_pos={0 0}, upload=55603]: Done begin upload segment. Jul 26 21:31:42.877896201 (57h55m52.600459241s) [noise=true , address=79.127.219.51:4212 , piece_id=JDWDNHN2XYUBNGB6ABIUOXXHE4VEHJ2OFLAEMHQCPLYMV6BIDJ4Q, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.878245138 (57h55m52.600808178s) [noise=true , address=104.36.82.53:4154 , piece_id=NNG7OQKHGOAGNV2FYV53IOBL6TV6L7CLTBIP73BPFORM5S2FBXTQ, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.878505667 (57h55m52.601068707s) [noise=true , address=104.36.82.40:4942 , piece_id=V62LH5T5LAS5NI6G6SK5HPRQ6JDEMVJTOZ2AVVGEG7BCVCGQV5UA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.878751487 (57h55m52.601314527s) [noise=true , address=156.146.43.225:4434 , piece_id=Z3TIFIZIFST7NRTDKDMYNJ447SYSUHXJXKTFI2LAZHSDLSUKQPDQ, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.878866617 (57h55m52.601429657s) [noise=true , address=104.36.82.19:4432 , piece_id=ZUUGGWLNS43YOKP3W3BNNKTQQ4TOA4TE4344MTKBSFYE2Q62ZUEA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.879137065 (57h55m52.601700105s) [noise=true , address=79.127.205.235:4690 , piece_id=N2Q4HYYSRIS4JRGL5DAZ2ADXB467VVI5ZLSYNVC7LSO4ETE26EEA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.879395750 (57h55m52.60195879s) [noise=true , address=104.36.82.54:4157 , piece_id=S6RHDQG3VWDGKJPKVF45EKGHO5SXWTEIBI2V6ZHUQ3B34HPTBH3A, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.879646510 (57h55m52.60220955s) [noise=true , address=79.127.201.215:4353 , piece_id=EYNR6JASXG6O2WDYN7VD4MMXO2XLPPMSX4VMK6BUQCRIMDVZ643Q, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.879886479 (57h55m52.602449519s) [noise=true , address=79.127.205.235:4668 , piece_id=7BSYNQMPKLU7I6UDT37ZC5JYBLXUQZ4N6FP6NFWZIYJW5O4TUICA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.880142409 (57h55m52.602705449s) [noise=true , address=79.127.219.38:4017 , piece_id=LTECTKX4EGOSD6CIU2UJTSF6XPJHKRZG7XVEU5NQ7GOPBZKMCZJA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.880437373 (57h55m52.603000413s) [noise=true , address=104.36.82.21:4485 , piece_id=CX4HTMIQGU5SWTIOAOIHRXGN6PGKTANEZHKPSPWSK4MF7HRHRRKQ, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.880674056 (57h55m52.603237096s) [noise=true , address=77.90.31.12:4611 , piece_id=PLCGHYXJQ6PQHZKNENCRAZU27VGHVTABUPMRJQ4AM4WLOH5DMYBA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.880924395 (57h55m52.603487435s) [noise=true , address=104.36.82.47:4544 , piece_id=VLWF7DLE6KP7UEBMKS2TYUKZ2KORYLHVZJTQG7Q7W7V6Y3MQR3UQ, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.881203739 (57h55m52.603766779s) [noise=true , address=104.36.82.7:4932 , piece_id=N4TIHZ5CM227SPM3CLVBQMDH6KJU5VYLXO25RFQEZQBYYHTYWQ6Q, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.881386870 (57h55m52.60394991s) [noise=true , address=104.36.82.46:4490 , piece_id=B55RR4PNTCE7JWPQDD3HG2VGLFLO23ETHH3I4ENJC3ROWPB3AVHQ, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.881674010 (57h55m52.60423705s) [noise=true , address=104.36.82.18:4377 , piece_id=5E3BXUBYCO73CUGZPOK4RNCGGE2V24FKAT6PFJKEAUAUEXCF34IA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.881891765 (57h55m52.604454805s) [noise=true , address=104.36.82.52:4090 , piece_id=VGTG37EHM5S5E5O7VG4JDNNBB6BHATWYMCCPLF7W2KBYL5U7BQAA, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:42.882117538 (57h55m52.604680578s) [noise=true , address=104.36.82.49:4003 , piece_id=JO2C2IQLDLNSEZDGUCE7KCKUUVG7KT2J3XGA6OO6KG64Q5OYEBCQ, seg_pos={0 0}, upload=55603]: Uploading piece... Jul 26 21:31:43.968038773 (57h55m53.690601813s) [upload=55603]: Next returned nil segment Jul 26 21:31:43.968054642 (57h55m53.690617683s) [upload=55603]: Waiting for error group managing segments... Jul 26 21:31:44.273936012 (57h55m53.996499052s) [noise=true , address=79.127.205.235:4668 , piece_id=7BSYNQMPKLU7I6UDT37ZC5JYBLXUQZ4N6FP6NFWZIYJW5O4TUICA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:44.275174291 (57h55m53.997737331s) [noise=true , address=79.127.205.235:4690 , piece_id=N2Q4HYYSRIS4JRGL5DAZ2ADXB467VVI5ZLSYNVC7LSO4ETE26EEA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:44.730769501 (57h55m54.453332551s) [noise=true , address=156.146.43.225:4434 , piece_id=Z3TIFIZIFST7NRTDKDMYNJ447SYSUHXJXKTFI2LAZHSDLSUKQPDQ, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:44.830817555 (57h55m54.553380595s) [noise=true , address=79.127.201.215:4353 , piece_id=EYNR6JASXG6O2WDYN7VD4MMXO2XLPPMSX4VMK6BUQCRIMDVZ643Q, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:45.154398949 (57h55m54.876961989s) [noise=true , address=79.127.219.51:4212 , piece_id=JDWDNHN2XYUBNGB6ABIUOXXHE4VEHJ2OFLAEMHQCPLYMV6BIDJ4Q, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:45.343174698 (57h55m55.065737738s) [noise=true , address=79.127.219.38:4017 , piece_id=LTECTKX4EGOSD6CIU2UJTSF6XPJHKRZG7XVEU5NQ7GOPBZKMCZJA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:51.342716234 (57h56m1.065279274s) [noise=true , address=104.36.82.18:4377 , piece_id=5E3BXUBYCO73CUGZPOK4RNCGGE2V24FKAT6PFJKEAUAUEXCF34IA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:31:52.859185965 (57h56m2.581748995s) [noise=true , address=77.90.31.12:4611 , piece_id=PLCGHYXJQ6PQHZKNENCRAZU27VGHVTABUPMRJQ4AM4WLOH5DMYBA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:32:03.086610257 (57h56m12.809173297s) [noise=true , address=104.36.82.21:4485 , piece_id=CX4HTMIQGU5SWTIOAOIHRXGN6PGKTANEZHKPSPWSK4MF7HRHRRKQ, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:32:05.040724663 (57h56m14.763287703s) [noise=true , address=104.36.82.49:4003 , piece_id=JO2C2IQLDLNSEZDGUCE7KCKUUVG7KT2J3XGA6OO6KG64Q5OYEBCQ, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:32:19.068240946 (57h56m28.790803986s) [noise=true , address=104.36.82.7:4932 , piece_id=N4TIHZ5CM227SPM3CLVBQMDH6KJU5VYLXO25RFQEZQBYYHTYWQ6Q, seg_pos={0 0}, upload=55603]: Done uploading piece. err: <nil> Jul 26 21:32:41.781117582 (57h56m51.503680612s) [noise=true , address=104.36.82.40:4942 , piece_id=V62LH5T5LAS5NI6G6SK5HPRQ6JDEMVJTOZ2AVVGEG7BCVCGQV5UA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:104.36.82.40:4942): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:32:41.781454416 (57h56m51.504017456s) [noise=true , address=104.36.82.46:4490 , piece_id=B55RR4PNTCE7JWPQDD3HG2VGLFLO23ETHH3I4ENJC3ROWPB3AVHQ, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:104.36.82.46:4490): protocol: rpc: dial tcp 104.36.82.46:4490: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:41.781490846 (57h56m51.504053886s) [noise=true , address=104.36.82.52:4090 , piece_id=VGTG37EHM5S5E5O7VG4JDNNBB6BHATWYMCCPLF7W2KBYL5U7BQAA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:12UUnoD9p5FC7rq9fMSzu1phkFpsnPsu4UYxC59MhpMjJ6wUC6F, address:104.36.82.52:4090): protocol: rpc: dial tcp 104.36.82.52:4090: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:41.781526725 (57h56m51.504089765s) [noise=true , address=104.36.82.47:4544 , piece_id=VLWF7DLE6KP7UEBMKS2TYUKZ2KORYLHVZJTQG7Q7W7V6Y3MQR3UQ, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:12RSxwxw1WXkNcu7b3dEgJhyNkFGYUfKF4QVKmK323L5g3EKFnG, address:104.36.82.47:4544): protocol: rpc: dial tcp 104.36.82.47:4544: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:41.781555069 (57h56m51.504118109s) [noise=true , address=104.36.82.53:4154 , piece_id=NNG7OQKHGOAGNV2FYV53IOBL6TV6L7CLTBIP73BPFORM5S2FBXTQ, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:1vbGunXCRMkmfvBqHPmbuW7fBLKck4Vh9MGsX1q7P6VfX6DKqB, address:104.36.82.53:4154): protocol: rpc: dial tcp 104.36.82.53:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:41.781583153 (57h56m51.504146183s) [noise=true , address=104.36.82.19:4432 , piece_id=ZUUGGWLNS43YOKP3W3BNNKTQQ4TOA4TE4344MTKBSFYE2Q62ZUEA, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:12oh9XZaiLhaZdMyq1cx88yrrG9C9sHtH3RTA6W2Ly39t23ArSQ, address:104.36.82.19:4432): protocol: rpc: dial tcp 104.36.82.19:4432: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:41.781614573 (57h56m51.504177613s) [noise=true , address=104.36.82.54:4157 , piece_id=S6RHDQG3VWDGKJPKVF45EKGHO5SXWTEIBI2V6ZHUQ3B34HPTBH3A, seg_pos={0 0}, upload=55603]: Done uploading piece. err: ecclient: upload failed (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, address:104.36.82.54:4157): protocol: write tcp 188.166.225.170:58496->104.36.82.54:4157: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:32:41.781626446 (57h56m51.504189486s) [seg_pos={0 0}, upload=55603]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 26 21:32:42.017861519 (57h56m51.740424609s) [upload=55603]: Done uploading segments... Jul 26 21:32:42.017890615 (57h56m51.740453685s) [upload=55603]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration:* 1m0.238730387s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.........X...............] duration: 1.868964656s FAIL (2025-07-26 21:32:42 UTC) orbiter warning: uplink: too many requests Duration: 1.869025963s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 9.776436285s FAIL (2025-07-26 21:32:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== true , address=104.36.82.7:4912 , piece_id=Q65VVVBKTZVJLOBYM54V4HXFL4S7C3KCPYIU5NKC4YTB4SAKP5UA, seg_pos={0 0}, upload=55548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Jul 26 21:28:03.453885956 (57h52m13.176449046s) [seg_pos={0 0}, upload=55548]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:28:03.453992971 (57h52m13.176556061s) [upload=55548]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:28:03.454001949 (57h52m13.176565029s) [upload=55548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:28:03.877602712 (57h52m13.600165753s) [upload=55548]: Done uploading segments... Jul 26 21:28:03.877620878 (57h52m13.600183918s) [upload=55548]: Upload finished. err: <nil> Jul 26 21:29:12.291363619 (57h53m22.013926659s) [upload=55565]: Starting upload Jul 26 21:29:12.291424846 (57h53m22.013987886s) [upload=55565]: Done starting upload Jul 26 21:29:12.291476836 (57h53m22.014039886s) [upload=55565]: Uploading segments... Jul 26 21:29:12.291618597 (57h53m22.014181647s) [upload=55565]: Got next segment. Inline: false Jul 26 21:29:12.291643475 (57h53m22.014206525s) [upload=55565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:29:12.291648033 (57h53m22.014211073s) [upload=55565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:29:12.525464730 (57h53m22.24802777s) [seg_pos={0 0}, upload=55565]: Begin upload segment... Jul 26 21:29:12.525505949 (57h53m22.248068989s) [seg_pos={0 0}, upload=55565]: Done begin upload segment. Jul 26 21:29:12.525536969 (57h53m22.248100009s) [noise=true , address=45.91.116.13:4153 , piece_id=77UYAH7WPPQ6RJWRRCYOY7LRSMRHIR7WFYLR4EP65DMCKNQLYNLA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.525842052 (57h53m22.248405092s) [noise=true , address=77.90.31.16:4735 , piece_id=DKVQY5IZWS5AKEVM4R66VUOAPXSFUKPS5GRVY74IMYUEIRPXIUTA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526144571 (57h53m22.248707611s) [noise=true , address=45.91.116.13:4154 , piece_id=PUSNHJTSVQKWEBAOJNQBJZZ6GMTLDFYVPTNHDRR5DZRVZAZHADNA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526413305 (57h53m22.248976346s) [noise=true , address=45.91.116.11:4118 , piece_id=LZW5XT2CFQVX5WHZ5JGADQKGJK7K5ODW54DJAU5BRREM735TNZGA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526663254 (57h53m22.249226294s) [noise=true , address=104.36.82.15:4259 , piece_id=VYYIUHONG3ZOD72D7RHEQJ25LEFIJAHVQ2O346BTZBMEHEJE46TA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526908313 (57h53m22.249471353s) [noise=true , address=45.91.116.12:4227 , piece_id=L62H3UUWVBZF55L3AUJLG2QXS27QF4NOC6TJ2ISOLLTHVF5V5LSA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527171125 (57h53m22.249734165s) [noise=true , address=45.91.116.14:4190 , piece_id=EOZLRX5S3QXA5RJCFCNDHMNWUGV36TVHBT2WDFEP6T63HFHXRYYA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527426855 (57h53m22.249989895s) [noise=true , address=45.91.116.8:4048 , piece_id=4OLBHXX4RX5KNA76QZ2HXQGSO5STSJJCYRGYX6N7MU7AS7MYCDYA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527633279 (57h53m22.250196309s) [noise=true , address=45.91.116.6:4973 , piece_id=S2W3CP33U5GOOK674MRIESPKLBMC3WDXV4UZ4SCRUUFF5OWLXVKQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.530732579 (57h53m22.25329562s) [noise=true , address=104.36.82.12:4151 , piece_id=ASR2C7JW55LZ52KVMAOZLQM6NABWVHUFJT2TPHPTU47HWH53MBCQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531107997 (57h53m22.253671027s) [noise=true , address=45.91.116.10:4081 , piece_id=QBI5VXR5ZGPVXESAPL7VYXVSPZOABLL5CAECA2CNDO57VF7XXKWQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531380879 (57h53m22.253943919s) [noise=true , address=45.91.116.7:4011 , piece_id=NHSEPU764CIJKJACYKK3HUDVKAFBQD4CYDT57YJSTZWA3VFBGQAQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531653471 (57h53m22.254216511s) [noise=true , address=45.91.116.4:4940 , piece_id=IQPHTBFUGHJZJ6OOPG2TITFJYHW3TS43ITPJTB5BCQPIL5DTZ7QQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531957172 (57h53m22.254520212s) [noise=true , address=77.90.31.14:4661 , piece_id=Q6L7KYUJAOQVLZ3MNMF5CTVYTRIUIL7N3IG76MB33MGZ4A5E7Q7Q, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.532259030 (57h53m22.25482207s) [noise=true , address=104.36.82.5:4838 , piece_id=VD7MSDATYKFSHNII2BQNUM5CJELV2WVVLFFABDMNLNK4SQRUMZJQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.532528335 (57h53m22.255091375s) [noise=true , address=104.36.82.4:4801 , piece_id=UE2C7GSFFLE2FE6RZDDHW5WN2FLX4QB66XPMINYHQHZOTKMHQ5YA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:14.249462781 (57h53m23.972025821s) [upload=55565]: Next returned nil segment Jul 26 21:29:14.249490965 (57h53m23.972054005s) [upload=55565]: Waiting for error group managing segments... Jul 26 21:29:14.525576625 (57h53m24.248139665s) [noise=true , address=45.91.116.13:4153 , piece_id=77UYAH7WPPQ6RJWRRCYOY7LRSMRHIR7WFYLR4EP65DMCKNQLYNLA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.557616257 (57h53m24.280179297s) [noise=true , address=45.91.116.4:4940 , piece_id=IQPHTBFUGHJZJ6OOPG2TITFJYHW3TS43ITPJTB5BCQPIL5DTZ7QQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.619476866 (57h53m24.342039906s) [noise=true , address=45.91.116.6:4973 , piece_id=S2W3CP33U5GOOK674MRIESPKLBMC3WDXV4UZ4SCRUUFF5OWLXVKQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.619985460 (57h53m24.3425485s) [noise=true , address=45.91.116.8:4048 , piece_id=4OLBHXX4RX5KNA76QZ2HXQGSO5STSJJCYRGYX6N7MU7AS7MYCDYA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.628053045 (57h53m24.350616085s) [noise=true , address=45.91.116.12:4227 , piece_id=L62H3UUWVBZF55L3AUJLG2QXS27QF4NOC6TJ2ISOLLTHVF5V5LSA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.642985626 (57h53m24.365548666s) [noise=true , address=45.91.116.7:4011 , piece_id=NHSEPU764CIJKJACYKK3HUDVKAFBQD4CYDT57YJSTZWA3VFBGQAQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.647742958 (57h53m24.370305999s) [noise=true , address=45.91.116.13:4154 , piece_id=PUSNHJTSVQKWEBAOJNQBJZZ6GMTLDFYVPTNHDRR5DZRVZAZHADNA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.653237842 (57h53m24.375800872s) [noise=true , address=45.91.116.14:4190 , piece_id=EOZLRX5S3QXA5RJCFCNDHMNWUGV36TVHBT2WDFEP6T63HFHXRYYA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.656914856 (57h53m24.379477897s) [noise=true , address=45.91.116.10:4081 , piece_id=QBI5VXR5ZGPVXESAPL7VYXVSPZOABLL5CAECA2CNDO57VF7XXKWQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.705503492 (57h53m24.428066532s) [noise=true , address=45.91.116.11:4118 , piece_id=LZW5XT2CFQVX5WHZ5JGADQKGJK7K5ODW54DJAU5BRREM735TNZGA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:21.963538357 (57h53m31.686101397s) [noise=true , address=77.90.31.16:4735 , piece_id=DKVQY5IZWS5AKEVM4R66VUOAPXSFUKPS5GRVY74IMYUEIRPXIUTA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:29.707336511 (57h53m39.429899551s) [noise=true , address=77.90.31.14:4661 , piece_id=Q6L7KYUJAOQVLZ3MNMF5CTVYTRIUIL7N3IG76MB33MGZ4A5E7Q7Q, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:41.647725056 (57h53m51.370288097s) [noise=true , address=104.36.82.12:4151 , piece_id=ASR2C7JW55LZ52KVMAOZLQM6NABWVHUFJT2TPHPTU47HWH53MBCQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:54132->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647759903 (57h53m51.370322933s) [noise=true , address=104.36.82.5:4838 , piece_id=VD7MSDATYKFSHNII2BQNUM5CJELV2WVVLFFABDMNLNK4SQRUMZJQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647795592 (57h53m51.370358632s) [noise=true , address=104.36.82.4:4801 , piece_id=UE2C7GSFFLE2FE6RZDDHW5WN2FLX4QB66XPMINYHQHZOTKMHQ5YA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647815850 (57h53m51.37037888s) [noise=true , address=104.36.82.15:4259 , piece_id=VYYIUHONG3ZOD72D7RHEQJ25LEFIJAHVQ2O346BTZBMEHEJE46TA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 188.166.225.170:51486->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647844084 (57h53m51.370407124s) [seg_pos={0 0}, upload=55565]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:29:41.878479232 (57h53m51.601042332s) [upload=55565]: Done uploading segments... Jul 26 21:29:41.878514910 (57h53m51.60107801s) [upload=55565]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.776466092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.....................X...................................] duration: 8.201609096s FAIL (2025-07-26 21:32:24 UTC) orbiter warning: digest mismatch: after 56c2e09ba15d0e699d08c0d58f3749f0fbc9398fb5fe0914f19c8c549edc5ebd != before dc381fcf7a830470aaaf14a12393caa67b6283bf63bfbc61c099113c1510e5f0 Duration: 8.201647048s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 8.789430208s FAIL (2025-07-26 21:32:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== true , address=104.36.82.7:4912 , piece_id=Q65VVVBKTZVJLOBYM54V4HXFL4S7C3KCPYIU5NKC4YTB4SAKP5UA, seg_pos={0 0}, upload=55548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Jul 26 21:28:03.453885956 (57h52m13.176449046s) [seg_pos={0 0}, upload=55548]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:28:03.453992971 (57h52m13.176556061s) [upload=55548]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:28:03.454001949 (57h52m13.176565029s) [upload=55548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:28:03.877602712 (57h52m13.600165753s) [upload=55548]: Done uploading segments... Jul 26 21:28:03.877620878 (57h52m13.600183918s) [upload=55548]: Upload finished. err: <nil> Jul 26 21:29:12.291363619 (57h53m22.013926659s) [upload=55565]: Starting upload Jul 26 21:29:12.291424846 (57h53m22.013987886s) [upload=55565]: Done starting upload Jul 26 21:29:12.291476836 (57h53m22.014039886s) [upload=55565]: Uploading segments... Jul 26 21:29:12.291618597 (57h53m22.014181647s) [upload=55565]: Got next segment. Inline: false Jul 26 21:29:12.291643475 (57h53m22.014206525s) [upload=55565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:29:12.291648033 (57h53m22.014211073s) [upload=55565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:29:12.525464730 (57h53m22.24802777s) [seg_pos={0 0}, upload=55565]: Begin upload segment... Jul 26 21:29:12.525505949 (57h53m22.248068989s) [seg_pos={0 0}, upload=55565]: Done begin upload segment. Jul 26 21:29:12.525536969 (57h53m22.248100009s) [noise=true , address=45.91.116.13:4153 , piece_id=77UYAH7WPPQ6RJWRRCYOY7LRSMRHIR7WFYLR4EP65DMCKNQLYNLA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.525842052 (57h53m22.248405092s) [noise=true , address=77.90.31.16:4735 , piece_id=DKVQY5IZWS5AKEVM4R66VUOAPXSFUKPS5GRVY74IMYUEIRPXIUTA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526144571 (57h53m22.248707611s) [noise=true , address=45.91.116.13:4154 , piece_id=PUSNHJTSVQKWEBAOJNQBJZZ6GMTLDFYVPTNHDRR5DZRVZAZHADNA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526413305 (57h53m22.248976346s) [noise=true , address=45.91.116.11:4118 , piece_id=LZW5XT2CFQVX5WHZ5JGADQKGJK7K5ODW54DJAU5BRREM735TNZGA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526663254 (57h53m22.249226294s) [noise=true , address=104.36.82.15:4259 , piece_id=VYYIUHONG3ZOD72D7RHEQJ25LEFIJAHVQ2O346BTZBMEHEJE46TA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526908313 (57h53m22.249471353s) [noise=true , address=45.91.116.12:4227 , piece_id=L62H3UUWVBZF55L3AUJLG2QXS27QF4NOC6TJ2ISOLLTHVF5V5LSA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527171125 (57h53m22.249734165s) [noise=true , address=45.91.116.14:4190 , piece_id=EOZLRX5S3QXA5RJCFCNDHMNWUGV36TVHBT2WDFEP6T63HFHXRYYA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527426855 (57h53m22.249989895s) [noise=true , address=45.91.116.8:4048 , piece_id=4OLBHXX4RX5KNA76QZ2HXQGSO5STSJJCYRGYX6N7MU7AS7MYCDYA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527633279 (57h53m22.250196309s) [noise=true , address=45.91.116.6:4973 , piece_id=S2W3CP33U5GOOK674MRIESPKLBMC3WDXV4UZ4SCRUUFF5OWLXVKQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.530732579 (57h53m22.25329562s) [noise=true , address=104.36.82.12:4151 , piece_id=ASR2C7JW55LZ52KVMAOZLQM6NABWVHUFJT2TPHPTU47HWH53MBCQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531107997 (57h53m22.253671027s) [noise=true , address=45.91.116.10:4081 , piece_id=QBI5VXR5ZGPVXESAPL7VYXVSPZOABLL5CAECA2CNDO57VF7XXKWQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531380879 (57h53m22.253943919s) [noise=true , address=45.91.116.7:4011 , piece_id=NHSEPU764CIJKJACYKK3HUDVKAFBQD4CYDT57YJSTZWA3VFBGQAQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531653471 (57h53m22.254216511s) [noise=true , address=45.91.116.4:4940 , piece_id=IQPHTBFUGHJZJ6OOPG2TITFJYHW3TS43ITPJTB5BCQPIL5DTZ7QQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531957172 (57h53m22.254520212s) [noise=true , address=77.90.31.14:4661 , piece_id=Q6L7KYUJAOQVLZ3MNMF5CTVYTRIUIL7N3IG76MB33MGZ4A5E7Q7Q, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.532259030 (57h53m22.25482207s) [noise=true , address=104.36.82.5:4838 , piece_id=VD7MSDATYKFSHNII2BQNUM5CJELV2WVVLFFABDMNLNK4SQRUMZJQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.532528335 (57h53m22.255091375s) [noise=true , address=104.36.82.4:4801 , piece_id=UE2C7GSFFLE2FE6RZDDHW5WN2FLX4QB66XPMINYHQHZOTKMHQ5YA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:14.249462781 (57h53m23.972025821s) [upload=55565]: Next returned nil segment Jul 26 21:29:14.249490965 (57h53m23.972054005s) [upload=55565]: Waiting for error group managing segments... Jul 26 21:29:14.525576625 (57h53m24.248139665s) [noise=true , address=45.91.116.13:4153 , piece_id=77UYAH7WPPQ6RJWRRCYOY7LRSMRHIR7WFYLR4EP65DMCKNQLYNLA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.557616257 (57h53m24.280179297s) [noise=true , address=45.91.116.4:4940 , piece_id=IQPHTBFUGHJZJ6OOPG2TITFJYHW3TS43ITPJTB5BCQPIL5DTZ7QQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.619476866 (57h53m24.342039906s) [noise=true , address=45.91.116.6:4973 , piece_id=S2W3CP33U5GOOK674MRIESPKLBMC3WDXV4UZ4SCRUUFF5OWLXVKQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.619985460 (57h53m24.3425485s) [noise=true , address=45.91.116.8:4048 , piece_id=4OLBHXX4RX5KNA76QZ2HXQGSO5STSJJCYRGYX6N7MU7AS7MYCDYA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.628053045 (57h53m24.350616085s) [noise=true , address=45.91.116.12:4227 , piece_id=L62H3UUWVBZF55L3AUJLG2QXS27QF4NOC6TJ2ISOLLTHVF5V5LSA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.642985626 (57h53m24.365548666s) [noise=true , address=45.91.116.7:4011 , piece_id=NHSEPU764CIJKJACYKK3HUDVKAFBQD4CYDT57YJSTZWA3VFBGQAQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.647742958 (57h53m24.370305999s) [noise=true , address=45.91.116.13:4154 , piece_id=PUSNHJTSVQKWEBAOJNQBJZZ6GMTLDFYVPTNHDRR5DZRVZAZHADNA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.653237842 (57h53m24.375800872s) [noise=true , address=45.91.116.14:4190 , piece_id=EOZLRX5S3QXA5RJCFCNDHMNWUGV36TVHBT2WDFEP6T63HFHXRYYA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.656914856 (57h53m24.379477897s) [noise=true , address=45.91.116.10:4081 , piece_id=QBI5VXR5ZGPVXESAPL7VYXVSPZOABLL5CAECA2CNDO57VF7XXKWQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.705503492 (57h53m24.428066532s) [noise=true , address=45.91.116.11:4118 , piece_id=LZW5XT2CFQVX5WHZ5JGADQKGJK7K5ODW54DJAU5BRREM735TNZGA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:21.963538357 (57h53m31.686101397s) [noise=true , address=77.90.31.16:4735 , piece_id=DKVQY5IZWS5AKEVM4R66VUOAPXSFUKPS5GRVY74IMYUEIRPXIUTA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:29.707336511 (57h53m39.429899551s) [noise=true , address=77.90.31.14:4661 , piece_id=Q6L7KYUJAOQVLZ3MNMF5CTVYTRIUIL7N3IG76MB33MGZ4A5E7Q7Q, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:41.647725056 (57h53m51.370288097s) [noise=true , address=104.36.82.12:4151 , piece_id=ASR2C7JW55LZ52KVMAOZLQM6NABWVHUFJT2TPHPTU47HWH53MBCQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:54132->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647759903 (57h53m51.370322933s) [noise=true , address=104.36.82.5:4838 , piece_id=VD7MSDATYKFSHNII2BQNUM5CJELV2WVVLFFABDMNLNK4SQRUMZJQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647795592 (57h53m51.370358632s) [noise=true , address=104.36.82.4:4801 , piece_id=UE2C7GSFFLE2FE6RZDDHW5WN2FLX4QB66XPMINYHQHZOTKMHQ5YA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647815850 (57h53m51.37037888s) [noise=true , address=104.36.82.15:4259 , piece_id=VYYIUHONG3ZOD72D7RHEQJ25LEFIJAHVQ2O346BTZBMEHEJE46TA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 188.166.225.170:51486->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647844084 (57h53m51.370407124s) [seg_pos={0 0}, upload=55565]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:29:41.878479232 (57h53m51.601042332s) [upload=55565]: Done uploading segments... Jul 26 21:29:41.878514910 (57h53m51.60107801s) [upload=55565]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.789448573s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 29.12406344s FAIL (2025-07-26 21:31:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== true , address=104.36.82.7:4912 , piece_id=Q65VVVBKTZVJLOBYM54V4HXFL4S7C3KCPYIU5NKC4YTB4SAKP5UA, seg_pos={0 0}, upload=55548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Jul 26 21:28:03.453885956 (57h52m13.176449046s) [seg_pos={0 0}, upload=55548]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:28:03.453992971 (57h52m13.176556061s) [upload=55548]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:28:03.454001949 (57h52m13.176565029s) [upload=55548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:28:03.877602712 (57h52m13.600165753s) [upload=55548]: Done uploading segments... Jul 26 21:28:03.877620878 (57h52m13.600183918s) [upload=55548]: Upload finished. err: <nil> Jul 26 21:29:12.291363619 (57h53m22.013926659s) [upload=55565]: Starting upload Jul 26 21:29:12.291424846 (57h53m22.013987886s) [upload=55565]: Done starting upload Jul 26 21:29:12.291476836 (57h53m22.014039886s) [upload=55565]: Uploading segments... Jul 26 21:29:12.291618597 (57h53m22.014181647s) [upload=55565]: Got next segment. Inline: false Jul 26 21:29:12.291643475 (57h53m22.014206525s) [upload=55565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:29:12.291648033 (57h53m22.014211073s) [upload=55565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:29:12.525464730 (57h53m22.24802777s) [seg_pos={0 0}, upload=55565]: Begin upload segment... Jul 26 21:29:12.525505949 (57h53m22.248068989s) [seg_pos={0 0}, upload=55565]: Done begin upload segment. Jul 26 21:29:12.525536969 (57h53m22.248100009s) [noise=true , address=45.91.116.13:4153 , piece_id=77UYAH7WPPQ6RJWRRCYOY7LRSMRHIR7WFYLR4EP65DMCKNQLYNLA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.525842052 (57h53m22.248405092s) [noise=true , address=77.90.31.16:4735 , piece_id=DKVQY5IZWS5AKEVM4R66VUOAPXSFUKPS5GRVY74IMYUEIRPXIUTA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526144571 (57h53m22.248707611s) [noise=true , address=45.91.116.13:4154 , piece_id=PUSNHJTSVQKWEBAOJNQBJZZ6GMTLDFYVPTNHDRR5DZRVZAZHADNA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526413305 (57h53m22.248976346s) [noise=true , address=45.91.116.11:4118 , piece_id=LZW5XT2CFQVX5WHZ5JGADQKGJK7K5ODW54DJAU5BRREM735TNZGA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526663254 (57h53m22.249226294s) [noise=true , address=104.36.82.15:4259 , piece_id=VYYIUHONG3ZOD72D7RHEQJ25LEFIJAHVQ2O346BTZBMEHEJE46TA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.526908313 (57h53m22.249471353s) [noise=true , address=45.91.116.12:4227 , piece_id=L62H3UUWVBZF55L3AUJLG2QXS27QF4NOC6TJ2ISOLLTHVF5V5LSA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527171125 (57h53m22.249734165s) [noise=true , address=45.91.116.14:4190 , piece_id=EOZLRX5S3QXA5RJCFCNDHMNWUGV36TVHBT2WDFEP6T63HFHXRYYA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527426855 (57h53m22.249989895s) [noise=true , address=45.91.116.8:4048 , piece_id=4OLBHXX4RX5KNA76QZ2HXQGSO5STSJJCYRGYX6N7MU7AS7MYCDYA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.527633279 (57h53m22.250196309s) [noise=true , address=45.91.116.6:4973 , piece_id=S2W3CP33U5GOOK674MRIESPKLBMC3WDXV4UZ4SCRUUFF5OWLXVKQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.530732579 (57h53m22.25329562s) [noise=true , address=104.36.82.12:4151 , piece_id=ASR2C7JW55LZ52KVMAOZLQM6NABWVHUFJT2TPHPTU47HWH53MBCQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531107997 (57h53m22.253671027s) [noise=true , address=45.91.116.10:4081 , piece_id=QBI5VXR5ZGPVXESAPL7VYXVSPZOABLL5CAECA2CNDO57VF7XXKWQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531380879 (57h53m22.253943919s) [noise=true , address=45.91.116.7:4011 , piece_id=NHSEPU764CIJKJACYKK3HUDVKAFBQD4CYDT57YJSTZWA3VFBGQAQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531653471 (57h53m22.254216511s) [noise=true , address=45.91.116.4:4940 , piece_id=IQPHTBFUGHJZJ6OOPG2TITFJYHW3TS43ITPJTB5BCQPIL5DTZ7QQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.531957172 (57h53m22.254520212s) [noise=true , address=77.90.31.14:4661 , piece_id=Q6L7KYUJAOQVLZ3MNMF5CTVYTRIUIL7N3IG76MB33MGZ4A5E7Q7Q, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.532259030 (57h53m22.25482207s) [noise=true , address=104.36.82.5:4838 , piece_id=VD7MSDATYKFSHNII2BQNUM5CJELV2WVVLFFABDMNLNK4SQRUMZJQ, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:12.532528335 (57h53m22.255091375s) [noise=true , address=104.36.82.4:4801 , piece_id=UE2C7GSFFLE2FE6RZDDHW5WN2FLX4QB66XPMINYHQHZOTKMHQ5YA, seg_pos={0 0}, upload=55565]: Uploading piece... Jul 26 21:29:14.249462781 (57h53m23.972025821s) [upload=55565]: Next returned nil segment Jul 26 21:29:14.249490965 (57h53m23.972054005s) [upload=55565]: Waiting for error group managing segments... Jul 26 21:29:14.525576625 (57h53m24.248139665s) [noise=true , address=45.91.116.13:4153 , piece_id=77UYAH7WPPQ6RJWRRCYOY7LRSMRHIR7WFYLR4EP65DMCKNQLYNLA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.557616257 (57h53m24.280179297s) [noise=true , address=45.91.116.4:4940 , piece_id=IQPHTBFUGHJZJ6OOPG2TITFJYHW3TS43ITPJTB5BCQPIL5DTZ7QQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.619476866 (57h53m24.342039906s) [noise=true , address=45.91.116.6:4973 , piece_id=S2W3CP33U5GOOK674MRIESPKLBMC3WDXV4UZ4SCRUUFF5OWLXVKQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.619985460 (57h53m24.3425485s) [noise=true , address=45.91.116.8:4048 , piece_id=4OLBHXX4RX5KNA76QZ2HXQGSO5STSJJCYRGYX6N7MU7AS7MYCDYA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.628053045 (57h53m24.350616085s) [noise=true , address=45.91.116.12:4227 , piece_id=L62H3UUWVBZF55L3AUJLG2QXS27QF4NOC6TJ2ISOLLTHVF5V5LSA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.642985626 (57h53m24.365548666s) [noise=true , address=45.91.116.7:4011 , piece_id=NHSEPU764CIJKJACYKK3HUDVKAFBQD4CYDT57YJSTZWA3VFBGQAQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.647742958 (57h53m24.370305999s) [noise=true , address=45.91.116.13:4154 , piece_id=PUSNHJTSVQKWEBAOJNQBJZZ6GMTLDFYVPTNHDRR5DZRVZAZHADNA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.653237842 (57h53m24.375800872s) [noise=true , address=45.91.116.14:4190 , piece_id=EOZLRX5S3QXA5RJCFCNDHMNWUGV36TVHBT2WDFEP6T63HFHXRYYA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.656914856 (57h53m24.379477897s) [noise=true , address=45.91.116.10:4081 , piece_id=QBI5VXR5ZGPVXESAPL7VYXVSPZOABLL5CAECA2CNDO57VF7XXKWQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:14.705503492 (57h53m24.428066532s) [noise=true , address=45.91.116.11:4118 , piece_id=LZW5XT2CFQVX5WHZ5JGADQKGJK7K5ODW54DJAU5BRREM735TNZGA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:21.963538357 (57h53m31.686101397s) [noise=true , address=77.90.31.16:4735 , piece_id=DKVQY5IZWS5AKEVM4R66VUOAPXSFUKPS5GRVY74IMYUEIRPXIUTA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:29.707336511 (57h53m39.429899551s) [noise=true , address=77.90.31.14:4661 , piece_id=Q6L7KYUJAOQVLZ3MNMF5CTVYTRIUIL7N3IG76MB33MGZ4A5E7Q7Q, seg_pos={0 0}, upload=55565]: Done uploading piece. err: <nil> Jul 26 21:29:41.647725056 (57h53m51.370288097s) [noise=true , address=104.36.82.12:4151 , piece_id=ASR2C7JW55LZ52KVMAOZLQM6NABWVHUFJT2TPHPTU47HWH53MBCQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:54132->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647759903 (57h53m51.370322933s) [noise=true , address=104.36.82.5:4838 , piece_id=VD7MSDATYKFSHNII2BQNUM5CJELV2WVVLFFABDMNLNK4SQRUMZJQ, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647795592 (57h53m51.370358632s) [noise=true , address=104.36.82.4:4801 , piece_id=UE2C7GSFFLE2FE6RZDDHW5WN2FLX4QB66XPMINYHQHZOTKMHQ5YA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: rpc: dial tcp 104.36.82.4:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647815850 (57h53m51.37037888s) [noise=true , address=104.36.82.15:4259 , piece_id=VYYIUHONG3ZOD72D7RHEQJ25LEFIJAHVQ2O346BTZBMEHEJE46TA, seg_pos={0 0}, upload=55565]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 188.166.225.170:51486->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:29:41.647844084 (57h53m51.370407124s) [seg_pos={0 0}, upload=55565]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:29:41.878479232 (57h53m51.601042332s) [upload=55565]: Done uploading segments... Jul 26 21:29:41.878514910 (57h53m51.60107801s) [upload=55565]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 29.124081605s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.....................X..................................] duration: 8.284442794s FAIL (2025-07-26 21:31:51 UTC) orbiter warning: digest mismatch: after 56c2e09ba15d0e699d08c0d58f3749f0fbc9398fb5fe0914f19c8c549edc5ebd != before dc381fcf7a830470aaaf14a12393caa67b6283bf63bfbc61c099113c1510e5f0 Duration: 8.284464876s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.........X..............] duration: 2.055186751s FAIL (2025-07-26 21:31:35 UTC) orbiter warning: uplink: too many requests Duration: 2.055219253s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.234506544s FAIL (2025-07-26 21:31:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== l 26 21:30:30.093339253 (57h54m39.815902293s) [noise=true , address=104.36.82.35:4594 , piece_id=WKPFV6OLRSAMNHUQTGUF3OMFGADO4DSC5ODCFPZR4Q2SSEKZC7TA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.093581597 (57h54m39.816144637s) [noise=true , address=104.36.82.19:4406 , piece_id=HJXFTREEKYEN5XCAXKWUG26ZUL5EFHLDZEK3G2RSQMLADLOWXGAA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.093783383 (57h54m39.816346423s) [noise=true , address=79.127.205.233:4650 , piece_id=J3G3TXCJGFQTVMCDRJTZHCOKXBCFDE6FDTS65QBZH4LC3SAEUXSA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.094062177 (57h54m39.816625217s) [noise=true , address=104.36.82.49:4977 , piece_id=VKYVFA7MYEESEWSOGXNJMU3AAYUOHZKTWRKJFISXXIPLUIVOFDQQ, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.094359054 (57h54m39.816922094s) [noise=true , address=79.127.205.245:4231 , piece_id=I4STLNLAOSLSWPDKH6LIUTPGNVHPZMXYVTSDGAOM7FAZ3CAQYFPQ, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.094609564 (57h54m39.817172604s) [noise=true , address=104.36.82.56:4244 , piece_id=XIZIXE4OBVOBANECIQM2CUV7WGKSMWEMGXAZBPTKVI3QSGZMCAQA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.094794858 (57h54m39.817357898s) [noise=true , address=104.36.82.15:4271 , piece_id=2JQOR3SOWRUPLX7V5IG3C754RCHGPYJOYTLTBPY2EWER6HXDSOOA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.095004850 (57h54m39.81756789s) [noise=true , address=104.36.82.54:4185 , piece_id=WJZR4LYXLIRUHIOAIZ6ZQGNWN3TVQAKMGRBSR5GQWHWTBJGERJWA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.095189043 (57h54m39.817752083s) [noise=true , address=104.36.82.14:4224 , piece_id=T2KHUJLHQVK5SOLPBDXZ3A62POT7LJVIWBVT6ZELJDPBD67M6ANQ, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.095374327 (57h54m39.817937367s) [noise=true , address=104.36.82.41:4794 , piece_id=HEOTWBHEJMALVOE2NEDWSQZADMS76OXOQ3ZGMZ23UXQ5YAQ3CMGA, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.095593807 (57h54m39.818156847s) [noise=true , address=104.36.82.53:4154 , piece_id=FLWK3MITJNY6P7ZSCUXI6ZMK62HAMOA4TTDSAZ7HO42GCJ6KELEQ, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.095775325 (57h54m39.818338355s) [noise=true , address=104.36.82.2:4752 , piece_id=EY7N3GA7UXT4FAZ7PQZLPSAUYEWQ6AY2NHE7RYFGBU2WUYWVSEVQ, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:30.095956190 (57h54m39.81851923s) [noise=true , address=79.127.205.234:4589 , piece_id=LUXABZ2XEL4S7REWKXBMNSNWKSVSJJ5TGJV6S6E6LKBFNIO7B4OQ, seg_pos={0 0}, upload=55586]: Uploading piece... Jul 26 21:30:31.187769399 (57h54m40.910332429s) [upload=55586]: Next returned nil segment Jul 26 21:30:31.187788266 (57h54m40.910351306s) [upload=55586]: Waiting for error group managing segments... Jul 26 21:30:31.477896355 (57h54m41.200459395s) [noise=true , address=79.127.205.234:4589 , piece_id=LUXABZ2XEL4S7REWKXBMNSNWKSVSJJ5TGJV6S6E6LKBFNIO7B4OQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: <nil> Jul 26 21:30:31.502402824 (57h54m41.224965864s) [noise=true , address=79.127.205.245:4231 , piece_id=I4STLNLAOSLSWPDKH6LIUTPGNVHPZMXYVTSDGAOM7FAZ3CAQYFPQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: <nil> Jul 26 21:30:31.517528353 (57h54m41.240091393s) [noise=true , address=79.127.205.233:4650 , piece_id=J3G3TXCJGFQTVMCDRJTZHCOKXBCFDE6FDTS65QBZH4LC3SAEUXSA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: <nil> Jul 26 21:30:35.318584763 (57h54m45.041147803s) [noise=true , address=79.127.201.215:4353 , piece_id=YVQ4HRSLPAM67K4PBQZT6V3RNJO5VVEOCPJFBBU6A7VVHAXOAS6Q, seg_pos={0 0}, upload=55586]: Done uploading piece. err: <nil> Jul 26 21:30:35.563551092 (57h54m45.286114132s) [noise=true , address=77.90.31.18:4823 , piece_id=AWRZ527SLDX43HTNEAQSMEQQJ5CXZFCGY6BS4C3RGHJJGSPWEBDQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: <nil> Jul 26 21:31:01.615682666 (57h55m11.338245706s) [noise=true , address=104.36.82.51:4061 , piece_id=RD6YPZR7DOBLGE45DKQXVZ3CYBL6TKDMUYQ2SQIQX7YA3BZZLEHQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: <nil> Jul 26 21:31:28.998365914 (57h55m38.720928944s) [noise=true , address=104.36.82.54:4185 , piece_id=WJZR4LYXLIRUHIOAIZ6ZQGNWN3TVQAKMGRBSR5GQWHWTBJGERJWA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:1cLdyyVLrp1yBmgsfBrfYc6N2RFbBichYoBbo2F7EZfaBXtkfk, address:104.36.82.54:4185): protocol: rpc: dial tcp 104.36.82.54:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998411722 (57h55m38.720974762s) [noise=true , address=104.36.82.14:4224 , piece_id=T2KHUJLHQVK5SOLPBDXZ3A62POT7LJVIWBVT6ZELJDPBD67M6ANQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:104.36.82.14:4224): protocol: rpc: dial tcp 104.36.82.14:4224: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998443754 (57h55m38.721006794s) [noise=true , address=104.36.82.41:4794 , piece_id=HEOTWBHEJMALVOE2NEDWSQZADMS76OXOQ3ZGMZ23UXQ5YAQ3CMGA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:12R7HqLyfJRftdvi2pD81NNrRVVnCEyKT9nrzv8eUWU4j2WGmzw, address:104.36.82.41:4794): protocol: rpc: dial tcp 104.36.82.41:4794: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998475765 (57h55m38.721038805s) [noise=true , address=104.36.82.35:4594 , piece_id=WKPFV6OLRSAMNHUQTGUF3OMFGADO4DSC5ODCFPZR4Q2SSEKZC7TA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: write tcp 188.166.225.170:40868->104.36.82.35:4594: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998504379 (57h55m38.721067419s) [noise=true , address=104.36.82.19:4406 , piece_id=HJXFTREEKYEN5XCAXKWUG26ZUL5EFHLDZEK3G2RSQMLADLOWXGAA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:104.36.82.19:4406): protocol: write tcp 188.166.225.170:59836->104.36.82.19:4406: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998531421 (57h55m38.721094461s) [noise=true , address=104.36.82.15:4271 , piece_id=2JQOR3SOWRUPLX7V5IG3C754RCHGPYJOYTLTBPY2EWER6HXDSOOA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:1T9uQjynbTYcaixhvHWCKzatRWpFxMonHFi5idiMQtvivupKSu, address:104.36.82.15:4271): protocol: rpc: dial tcp 104.36.82.15:4271: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998649949 (57h55m38.721212989s) [noise=true , address=104.36.82.2:4752 , piece_id=EY7N3GA7UXT4FAZ7PQZLPSAUYEWQ6AY2NHE7RYFGBU2WUYWVSEVQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:12oRYWQwoe5JtDCqfSShqtyRWVQLjL4UtbokBgNpDAocBKv1vqL, address:104.36.82.2:4752): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:31:28.998813161 (57h55m38.721376191s) [noise=true , address=104.36.82.49:4977 , piece_id=VKYVFA7MYEESEWSOGXNJMU3AAYUOHZKTWRKJFISXXIPLUIVOFDQQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:104.36.82.49:4977): protocol: write tcp 188.166.225.170:40610->104.36.82.49:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998859840 (57h55m38.72142288s) [noise=true , address=104.36.82.46:4490 , piece_id=6PGLNNOZVK2MXVTUW23TTAEIT5AREQBZEHO5MDO76QZO5BKVWTUA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji, address:104.36.82.46:4490): protocol: write tcp 188.166.225.170:44622->104.36.82.46:4490: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998890598 (57h55m38.721453638s) [noise=true , address=104.36.82.53:4154 , piece_id=FLWK3MITJNY6P7ZSCUXI6ZMK62HAMOA4TTDSAZ7HO42GCJ6KELEQ, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:1vbGunXCRMkmfvBqHPmbuW7fBLKck4Vh9MGsX1q7P6VfX6DKqB, address:104.36.82.53:4154): protocol: rpc: dial tcp 104.36.82.53:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.998981904 (57h55m38.721544944s) [noise=true , address=104.36.82.56:4244 , piece_id=XIZIXE4OBVOBANECIQM2CUV7WGKSMWEMGXAZBPTKVI3QSGZMCAQA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:12S1wh3PFQLPosjWmPs5iLfm4S5J6abXvoCfkhYz3KdvXCTaPRu, address:104.36.82.56:4244): protocol: write tcp 188.166.225.170:48816->104.36.82.56:4244: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:31:28.999051176 (57h55m38.721614216s) [noise=true , address=104.36.82.7:4922 , piece_id=N26HYZVZWZ4QQISQ6LO35JBVWXONXQIK6RH5VLKGZOCLLHOZQ5HA, seg_pos={0 0}, upload=55586]: Done uploading piece. err: ecclient: upload failed (node:12KrMgT4Pwj9UFE88q6tt65D7o6q68NhsmEXmifNmvzJ6eEkm8R, address:104.36.82.7:4922): protocol: write tcp 188.166.225.170:42460->104.36.82.7:4922: use of closed network connection; conn closed: op Jul 26 21:31:28.999063750 (57h55m38.72162679s) [seg_pos={0 0}, upload=55586]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:31:29.231246921 (57h55m38.953809961s) [upload=55586]: Done uploading segments... Jul 26 21:31:29.231272761 (57h55m38.953835801s) [upload=55586]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.234518887s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.........X.............] duration: 1.912007876s FAIL (2025-07-26 21:31:08 UTC) orbiter warning: uplink: too many requests Duration: 1.912035247s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)