Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X.................X.........] duration: 19.911023986s FAIL (2025-07-18 16:00:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 19.9110525s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 603.482441ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:00:22 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 603.507529ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.................X........] duration: 9.889647187s FAIL (2025-07-18 16:00:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 9.889672105s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 452.230738ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:59:57 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 452.257729ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.................X.......] duration: 5.788343024s FAIL (2025-07-18 15:59:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.788367953s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 445.04041ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:59:43 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 445.067022ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.................X......] duration: 5.906394499s FAIL (2025-07-18 15:59:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.906432682s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 447.890955ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:58:58 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 447.915532ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 708.221049ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:58:54 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 708.248773ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 708.960897ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:58:26 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 709.017686ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 574.79902ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:58:25 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 574.858684ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.................X.....] duration: 6.09991114s FAIL (2025-07-18 15:58:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.09993743s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 454.549663ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:58:05 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 454.573848ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.................X....] duration: 24.276683789s FAIL (2025-07-18 15:57:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 24.276716322s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 470.489792ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:57:44 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 470.514329ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 494.399541ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:57:33 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 494.421963ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 439.250982ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:57:24 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 439.300749ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 446.7289ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:57:05 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 446.752907ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.................X...] duration: 5.933007563s FAIL (2025-07-18 15:57:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.933052309s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 443.258449ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:57:00 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 443.276314ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.................X..] duration: 25.393402112s FAIL (2025-07-18 15:56:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 25.393429174s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 518.134833ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:56:30 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 518.15971ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 461.312675ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:56:22 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 461.334778ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 472.043184ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:55:37 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 472.076206ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 452.196711ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:55:22 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 452.228062ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.................X.] duration: 5.805632002s FAIL (2025-07-18 15:55:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.80571044s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.................X] duration: 30.244799041s FAIL (2025-07-18 15:54:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== h43m32.61326088s) [upload=485197]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:53:58.136222004 (606h43m32.906863379s) [upload=485197]: Done uploading segments... Jul 18 15:53:58.136239908 (606h43m32.906881243s) [upload=485197]: Upload finished. err: <nil> Jul 18 15:54:13.749706484 (606h43m48.520347879s) [upload=485200]: Starting upload Jul 18 15:54:13.749897359 (606h43m48.520538694s) [upload=485200]: Done starting upload Jul 18 15:54:13.749986089 (606h43m48.520627484s) [upload=485200]: Uploading segments... Jul 18 15:54:13.750348782 (606h43m48.520990158s) [upload=485200]: Got next segment. Inline: false Jul 18 15:54:13.750409819 (606h43m48.521051184s) [upload=485200]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:54:13.750418106 (606h43m48.521059521s) [upload=485200]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:54:13.985281107 (606h43m48.755922432s) [seg_pos={0 0}, upload=485200]: Begin upload segment... Jul 18 15:54:13.985320072 (606h43m48.755961407s) [seg_pos={0 0}, upload=485200]: Done begin upload segment. Jul 18 15:54:13.985345231 (606h43m48.755986566s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985359517 (606h43m48.756000852s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985692364 (606h43m48.756333699s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.985967671 (606h43m48.756609006s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986279368 (606h43m48.756920753s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986345294 (606h43m48.756986629s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986709751 (606h43m48.757351086s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.986974818 (606h43m48.757616153s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987186504 (606h43m48.757827839s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987297977 (606h43m48.757939312s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.987839543 (606h43m48.758480868s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988054383 (606h43m48.758695719s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988227405 (606h43m48.75886873s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988561374 (606h43m48.759202709s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988628181 (606h43m48.759269516s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:13.988929949 (606h43m48.759571274s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Uploading piece... Jul 18 15:54:15.790133555 (606h43m50.5607749s) [upload=485200]: Next returned nil segment Jul 18 15:54:15.790154626 (606h43m50.560795961s) [upload=485200]: Waiting for error group managing segments... Jul 18 15:54:16.100387544 (606h43m50.871028879s) [noise=true , address=104.36.82.13:4185 , piece_id=7TPRHARU3YTW76GOB7ARF7G4CKLULCGPG3HFQLLLRKVEVE4NRWBA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.100795494 (606h43m50.871436819s) [noise=true , address=104.36.82.2:4732 , piece_id=U7UL32S4FGDBFAO5NOZTJIPC35LYQCKFV52JAQ3JGVVWP3YQEOHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.119592148 (606h43m50.890233483s) [noise=true , address=45.91.116.13:4155 , piece_id=ZMUWGJKMKYCVYHYHHDW2T3G2VRTPBEIWIFOZFPAQWFWMPTOUCR4A, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.125952728 (606h43m50.896594063s) [noise=true , address=104.36.82.4:4804 , piece_id=3KX6OT4HLKUTSY3VZQSF4ZYOJA4CTNU72AG543GNVHAIBMYPR6UQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.137770619 (606h43m50.908411954s) [noise=true , address=45.91.116.8:4048 , piece_id=JV2QKKSGDIJWKS3GH62VWYFZMZEDB3JOMMUD4SGEGMLX7IOYMGUQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.138372010 (606h43m50.909013345s) [noise=true , address=45.91.116.6:4975 , piece_id=YYO7EQCPNFRYGGH25BOLC3H6DQNDVFMDVV45XELRGKFP3HG5YYNA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.150271438 (606h43m50.920912773s) [noise=true , address=45.91.116.5:4299 , piece_id=LPRSKRCZVN7XHZVKY27DPMYGRY3IXBHISGI7VIPHOURRWH6GVDRA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.179564095 (606h43m50.95020543s) [noise=true , address=104.36.82.6:4875 , piece_id=P5SEW6KQBQOKJSLNLKL2VX7QJWS5MDUCP2SGSYIZ7CKF3KU3S2ZQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.198210492 (606h43m50.968851827s) [noise=true , address=104.36.82.15:4258 , piece_id=DFJGBB6JGNPTYEAHFP344C5ASMLCPSZ3JL3DJ56LMFXMIPAZHGHA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:16.270667205 (606h43m51.04130854s) [noise=true , address=45.91.116.14:4191 , piece_id=H5GS4MBEBPW3YAJNLMWMJKENTXK5BRYRTTJ6TW5RPQ3YURKDEIPQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: <nil> Jul 18 15:54:43.102903034 (606h44m17.873544359s) [noise=true , address=77.90.31.13:4627 , piece_id=BXRLW7J4BXIO4LNGSG5NLZC4EBEMRJW7TS4OFFXICZZQRNAOORVQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102953491 (606h44m17.873594826s) [noise=true , address=77.90.31.18:4805 , piece_id=JVKESIV3AE4N26LDLEUCRZNFUQFLXTXJX3BFDAB53M7CMNFUMQMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44964->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.102980212 (606h44m17.873621537s) [noise=true , address=77.90.31.12:4589 , piece_id=UJ6YHQO7BSNKMCKFN3I4SMPFFPWR4ZCXE7HA3IGOOAU7KX5N5GMA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103104519 (606h44m17.873745854s) [noise=true , address=77.90.31.14:4663 , piece_id=SU2THRDAJXJ77XEEFRTLNPPD4FDSHKXV63PHYMVZED26P3SY4NQA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:41270->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103126371 (606h44m17.873767706s) [noise=true , address=77.90.31.13:4626 , piece_id=VVH75WMRYX3O7LJV6GYEXWMCD7LI6X3QO5HQPZAITGLEJKK6Q4WA, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:34420->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103152101 (606h44m17.873793436s) [noise=true , address=77.90.31.19:4842 , piece_id=FJWSLINTWVFJLJWJ7Q6FMOOT2HI7QZYAFJZ6NNU5OSWWTDUVWAFQ, seg_pos={0 0}, upload=485200]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:37212->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:54:43.103173371 (606h44m17.873814707s) [seg_pos={0 0}, upload=485200]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 15:54:43.346072539 (606h44m18.116713864s) [upload=485200]: Done uploading segments... Jul 18 15:54:43.346096544 (606h44m18.116737879s) [upload=485200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 30.244821483s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 459.970135ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:54:41 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 460.012868ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 467.210719ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:54:28 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 467.233884ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.26312769s FAIL (2025-07-18 15:54:00 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 6.263150776s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 447.32057ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:54:00 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 447.343926ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 9.830489474s FAIL (2025-07-18 15:53:32 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 9.830519911s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 450.972659ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:53:20 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 450.997115ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 26.69342623s FAIL (2025-07-18 15:53:11 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 26.693452921s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 466.224713ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:52:53 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 466.241474ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 475.452338ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:52:53 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 475.470533ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 461.224106ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:52:31 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 461.247209ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 462.599957ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:52:09 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 462.62268ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X............................................X..............] duration: 5.827774318s FAIL (2025-07-18 15:52:06 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 5.827800178s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 483.205652ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:51:59 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 483.223094ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 462.309411ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:51:46 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 462.333357ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 454.423121ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:51:27 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 454.444663ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX............................................X.............] duration: 5.778388036s FAIL (2025-07-18 15:50:53 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 5.77841575s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX............................................X............] duration: 6.026871061s FAIL (2025-07-18 15:50:32 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 6.026930265s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XX............................................X...........] duration: 25.691271932s FAIL (2025-07-18 15:49:59 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 25.691298553s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X..XX............................................X..........] duration: 23.780604275s FAIL (2025-07-18 15:48:52 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 23.780629704s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XX..XX............................................X.........] duration: 5.779198089s FAIL (2025-07-18 15:47:55 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 5.779224431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.XX..XX............................................X........] duration: 9.391467447s FAIL (2025-07-18 15:47:25 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 9.391489438s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..XX..XX............................................X.......] duration: 25.438618064s FAIL (2025-07-18 15:47:10 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 25.438643572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...XX..XX............................................X......] duration: 6.153795499s FAIL (2025-07-18 15:45:54 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Jul 18 15:39:18.375822865 (606h28m53.1464642s) [seg_pos={0 0}, upload=485007]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:39:18.375919660 (606h28m53.146560995s) [upload=485007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:39:18.375925361 (606h28m53.146566696s) [upload=485007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:39:18.668362273 (606h28m53.439003608s) [upload=485007]: Done uploading segments... Jul 18 15:39:18.668379757 (606h28m53.439021092s) [upload=485007]: Upload finished. err: <nil> Jul 18 15:39:46.384882970 (606h29m21.155524295s) [upload=485012]: Starting upload Jul 18 15:39:46.384937935 (606h29m21.15557927s) [upload=485012]: Done starting upload Jul 18 15:39:46.385028969 (606h29m21.155670304s) [upload=485012]: Uploading segments... Jul 18 15:39:46.385119492 (606h29m21.155760837s) [upload=485012]: Got next segment. Inline: false Jul 18 15:39:46.385167204 (606h29m21.155808539s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:39:46.385172023 (606h29m21.155813358s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:39:46.619972538 (606h29m21.390613873s) [seg_pos={0 0}, upload=485012]: Begin upload segment... Jul 18 15:39:46.620027153 (606h29m21.390668488s) [seg_pos={0 0}, upload=485012]: Done begin upload segment. Jul 18 15:39:46.620049095 (606h29m21.39069043s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620060115 (606h29m21.390701441s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620633823 (606h29m21.391275158s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.620986889 (606h29m21.391628214s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621251496 (606h29m21.391892831s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621525500 (606h29m21.392166845s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.621862935 (606h29m21.39250427s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622094187 (606h29m21.392735522s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622295734 (606h29m21.392937069s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622499433 (606h29m21.393140768s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622691210 (606h29m21.393332545s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.622938072 (606h29m21.393579407s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623127234 (606h29m21.393768569s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623311005 (606h29m21.39395234s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623505338 (606h29m21.394146673s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:46.623595230 (606h29m21.394236565s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Uploading piece... Jul 18 15:39:48.368504318 (606h29m23.139145653s) [upload=485012]: Next returned nil segment Jul 18 15:39:48.368525247 (606h29m23.139166582s) [upload=485012]: Waiting for error group managing segments... Jul 18 15:39:48.649330004 (606h29m23.419971339s) [noise=true , address=104.36.82.12:4151 , piece_id=DPK2AZQ6PV3ELRJSBCIG37SBX4I2CVOD7TJABCJPPGSAZZW3E3GQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.676285128 (606h29m23.446926463s) [noise=true , address=45.91.116.13:4154 , piece_id=5I2FFA56KQPU6AAX7CFKQTO4JWB46T2KP4RPU6BR3BPQGGXJG6KA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.681637599 (606h29m23.452278934s) [noise=true , address=45.91.116.13:4153 , piece_id=QX6VDGV32QUGT7RE6VOOO57NYJFSZAENSSVXTG3E3IHCWXLTSL2Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.685958977 (606h29m23.456600312s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2GS526YJUUP6CXOMDTROZ53HO6VVTGQXBMSVNTI6TDSF7JYJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.693711660 (606h29m23.464352985s) [noise=true , address=45.91.116.4:4937 , piece_id=BLD5TMVOWDPOOHMH3UTCV5EP4YT2THN7355ZVDKX6LVIC5JO7A6Q, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.698772633 (606h29m23.469413968s) [noise=true , address=104.36.82.8:4972 , piece_id=N5F72N5TYHGA3QFBQ7WAC6YTINI6Q6QUXKUDF7VL5DKWF77XPDHA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.752972274 (606h29m23.523613609s) [noise=true , address=45.91.116.5:4299 , piece_id=37NIXE2BRMDRULNDWHQ7S6JEZVYEAPQPY6QWDCAST4F4CEDO6V6A, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.755246294 (606h29m23.525887629s) [noise=true , address=104.36.82.6:4874 , piece_id=X3USMPPHCG7HWBD5VHOLZ3FUVHRQDLN6VQOH4Z5NZX7SYMS3LPLA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.765770160 (606h29m23.536411495s) [noise=true , address=45.91.116.6:4975 , piece_id=TFNFF2OWXPGCRRSSOXNVQAUV4C4WTGGR23PHXNE5OWTYY2GGM4XA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.802651084 (606h29m23.573292419s) [noise=true , address=45.91.116.11:4118 , piece_id=5U7BKJIXCZ7EB3TLKMY34IFNSKT4P7565O7UGPML7KPPCKFDJJGA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:39:48.830303582 (606h29m23.600944917s) [noise=true , address=104.36.82.9:4007 , piece_id=B3CC3EN3MF4OW6IUCSAER526B3OT3ZTFVQ7TMHWHEHFG5FLQRMKA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:11.826325645 (606h29m46.59696697s) [noise=true , address=77.90.31.19:4843 , piece_id=CGD7CMLIVIPDBCLN3374TF6GENY2E7B4PE66RGKFQQJIWXUNNAPA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216224398 (606h29m47.986865753s) [noise=true , address=77.90.31.21:4878 , piece_id=DV76NL2SCVORDCBYGO46RXLCSWLGLLXVBXP4LBXWFWTXPQZ6WPDA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: <nil> Jul 18 15:40:13.216247562 (606h29m47.986888917s) [seg_pos={0 0}, upload=485012]: Segment reached optimal threshold of 13 pieces. Jul 18 15:40:13.216495707 (606h29m47.987137042s) [noise=true , address=77.90.31.15:4698 , piece_id=N657IJIIUR72DQSYVR6M2HDW43675357ISRHPASF4TY7JYZLOWOA, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216571232 (606h29m47.987212557s) [noise=true , address=77.90.31.17:4771 , piece_id=2HJ47WD53CM3VBLEU44NBCVGI7ZEHM6RQZNVCTZER5D3WXPNRMXQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Jul 18 15:40:13.216657186 (606h29m47.987298521s) [noise=true , address=77.90.31.12:4589 , piece_id=S7NPOXVRZLGA6NIGH6YIO5BV4IMEL6QZNS5KMXGLBVUAEXJWIKVQ, seg_pos={0 0}, upload=485012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52698->77.90.31.12:4589: use of closed network connection Jul 18 15:40:13.216677555 (606h29m47.98731889s) [seg_pos={0 0}, upload=485012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:40:13.216760284 (606h29m47.987401619s) [upload=485012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:40:13.216764822 (606h29m47.987406177s) [upload=485012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:40:13.513062851 (606h29m48.283704186s) [upload=485012]: Done uploading segments... Jul 18 15:40:13.513083361 (606h29m48.283724696s) [upload=485012]: Upload finished. err: <nil> Duration: 6.153813123s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)