Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.464436731s FAIL (2025-07-09 03:44:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.464454886s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 27.995571773s FAIL (2025-07-09 03:44:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 27.995584117s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.991133815s FAIL (2025-07-09 03:43:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.991155305s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................................X] duration: 1m0.000979307s FAIL (2025-07-09 03:43:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000998002s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 8.643278735s FAIL (2025-07-09 03:43:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.643341636s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.115954205s FAIL (2025-07-09 03:42:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.115975424s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.753167296s FAIL (2025-07-09 03:42:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.753201922s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.493641737s FAIL (2025-07-09 03:42:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.493663238s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 8.04216631s FAIL (2025-07-09 03:42:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 8.042193033s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.502555499s FAIL (2025-07-09 03:41:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.502569415s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.817621523s FAIL (2025-07-09 03:41:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.817652973s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.874018979s FAIL (2025-07-09 03:41:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.874042355s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.554213882s FAIL (2025-07-09 03:41:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.554252677s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.671346979s FAIL (2025-07-09 03:40:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.671372908s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.883454614s FAIL (2025-07-09 03:40:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.883478239s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.797148434s FAIL (2025-07-09 03:40:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.7971724s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.071220243s FAIL (2025-07-09 03:40:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.071235311s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.51259781s FAIL (2025-07-09 03:40:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.512636454s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.04765585s FAIL (2025-07-09 03:39:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.047677732s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.099590036s FAIL (2025-07-09 03:39:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.099610105s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.746677496s FAIL (2025-07-09 03:39:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.746704187s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.51720795s FAIL (2025-07-09 03:39:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.517229941s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.002644872s FAIL (2025-07-09 03:38:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.002672825s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.815912204s FAIL (2025-07-09 03:38:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.815952721s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.513564754s FAIL (2025-07-09 03:38:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.513584903s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................X................] duration: 5.797910021s FAIL (2025-07-09 03:37:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.797934126s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X......X....................................................] duration: 5.522229523s FAIL (2025-07-09 03:37:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.522250943s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................................X...............] duration: 5.880771442s FAIL (2025-07-09 03:37:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.880798493s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.739548194s FAIL (2025-07-09 03:37:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.739575888s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [XX......X...................................................] duration: 5.569803767s FAIL (2025-07-09 03:36:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.569828685s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.X..........................................X..............] duration: 6.020670297s FAIL (2025-07-09 03:36:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.020691156s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [XXX......X..................................................] duration: 5.738957937s FAIL (2025-07-09 03:36:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.738979028s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.8607129s FAIL (2025-07-09 03:36:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.860744551s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.XXX......X.................................................] duration: 5.50921069s FAIL (2025-07-09 03:35:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.50923127s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.X..........................................X.............] duration: 6.094361286s FAIL (2025-07-09 03:35:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.094381256s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.785575866s FAIL (2025-07-09 03:35:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.785595663s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.X..........................................X............] duration: 6.369930908s FAIL (2025-07-09 03:35:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.369972258s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.551506424s FAIL (2025-07-09 03:34:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 6.551533737s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [X.XXX......X................................................] duration: 5.533075302s FAIL (2025-07-09 03:34:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.533090621s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.X.XXX......X...............................................] duration: 5.493378776s FAIL (2025-07-09 03:34:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.493398564s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.X..........................................X...........] duration: 6.001971123s FAIL (2025-07-09 03:34:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.00199571s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 8.923197168s FAIL (2025-07-09 03:34:14 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 8.923226182s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.X..........................................X..........] duration: 6.173060845s FAIL (2025-07-09 03:33:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.173073568s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..X.XXX......X..............................................] duration: 5.573461359s FAIL (2025-07-09 03:33:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.573474304s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.815436509s FAIL (2025-07-09 03:33:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.815467088s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [...X.XXX......X.............................................] duration: 36.507509397s FAIL (2025-07-09 03:33:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 36.507553312s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.X..........................................X.........] duration: 6.314350137s FAIL (2025-07-09 03:32:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.314417375s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [....X.XXX......X............................................] duration: 6.317837449s FAIL (2025-07-09 03:32:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.317886713s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.792116547s FAIL (2025-07-09 03:32:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== expected piece hash; context canceled; EOF Jul 9 03:00:26.821547853 (377h50m1.592189189s) [noise=true , address=45.91.116.7:4010 , piece_id=OR5D2DCY5B4IKLDVVZT73DXDUTABGQ4K4VDJEL5HBSV3SQSS4GCQ, seg_pos={0 0}, upload=302277]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Jul 9 03:00:26.821590575 (377h50m1.59223191s) [seg_pos={0 0}, upload=302277]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:00:26.821670989 (377h50m1.592312314s) [upload=302277]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:00:26.821675889 (377h50m1.592317224s) [upload=302277]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:00:27.129149168 (377h50m1.899790543s) [upload=302277]: Done uploading segments... Jul 9 03:00:27.129166300 (377h50m1.899807675s) [upload=302277]: Upload finished. err: <nil> Jul 9 03:01:31.849644782 (377h51m6.620286117s) [upload=302293]: Starting upload Jul 9 03:01:31.849706480 (377h51m6.620347815s) [upload=302293]: Done starting upload Jul 9 03:01:31.849908567 (377h51m6.620549902s) [upload=302293]: Uploading segments... Jul 9 03:01:31.849940148 (377h51m6.620581493s) [upload=302293]: Got next segment. Inline: false Jul 9 03:01:31.849961679 (377h51m6.620603014s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 03:01:31.849966318 (377h51m6.620607663s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 03:01:32.082815894 (377h51m6.853457229s) [seg_pos={0 0}, upload=302293]: Begin upload segment... Jul 9 03:01:32.082857022 (377h51m6.853498357s) [seg_pos={0 0}, upload=302293]: Done begin upload segment. Jul 9 03:01:32.082877130 (377h51m6.853518476s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.082910053 (377h51m6.853551378s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083224024 (377h51m6.853865359s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083306551 (377h51m6.853947887s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083473250 (377h51m6.854114585s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083576027 (377h51m6.854217352s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083720604 (377h51m6.854361929s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.083881712 (377h51m6.854523047s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084015208 (377h51m6.854656543s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084188880 (377h51m6.854830215s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084277228 (377h51m6.854918563s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084445321 (377h51m6.855086656s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084507439 (377h51m6.855148775s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084669670 (377h51m6.855311005s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084714255 (377h51m6.85535559s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:32.084930158 (377h51m6.855571493s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Uploading piece... Jul 9 03:01:33.718884639 (377h51m8.489525974s) [upload=302293]: Next returned nil segment Jul 9 03:01:33.718901952 (377h51m8.489543287s) [upload=302293]: Waiting for error group managing segments... Jul 9 03:01:34.050524955 (377h51m8.82116629s) [noise=true , address=104.36.82.11:4113 , piece_id=GPP2TC5NDK6GFNS4RHUUVQBKRLOLOBRQ2ZOYBWYB75ANW2EDGRRA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.058353302 (377h51m8.828994637s) [noise=true , address=104.36.82.2:4730 , piece_id=GSGFMMORJB3LEGHOBI7AHOOT64SLKLC3IBMMKGK4ZHET4APJ3HKA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.098151442 (377h51m8.868792777s) [noise=true , address=104.36.82.16:4295 , piece_id=BTGO6HOLM4QZT4ORXAYNDVXVVJY365HIHTIJ4DJC6MILW2WUXCRQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.110943775 (377h51m8.88158511s) [noise=true , address=104.36.82.9:4007 , piece_id=PWEBGYEIZKF7LMAP5OIPFHI4MUTEESH7AP4YNPR4HZRLQ6W2RK7A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.117461985 (377h51m8.88810332s) [noise=true , address=104.36.82.13:4185 , piece_id=ONP27OBE47FRF65P7WER6TPYUIQV5JL6HFPW27QZWWKIQRW4UZ6Q, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.161579608 (377h51m8.932220933s) [noise=true , address=45.91.116.5:4298 , piece_id=J6246HQ74DVP7OCDYP7RZV6LYMUOHSBZUHPGRRKIFFBMF6H2VJ4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170088417 (377h51m8.940729752s) [noise=true , address=45.91.116.4:4937 , piece_id=R2KT7NL4HIHVCAWL4VCHNZXJHD6LBLVLQT5VPTZNOIMHSSOKCFCA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.170743521 (377h51m8.941384846s) [noise=true , address=45.91.116.13:4154 , piece_id=ONJJYHN6V4GBSEJC6HQAVW6D4B3X3HMZYINJY3QTLPVH3CXDZXWQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.176248322 (377h51m8.946889647s) [noise=true , address=45.91.116.14:4189 , piece_id=C4T6UUDVQLCBC4ARQ5PYUTC2STW235PJVERKBETDJ33ZDEEZDCWA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.183619354 (377h51m8.954260679s) [noise=true , address=45.91.116.8:4045 , piece_id=5675275I45RIW7LNXNS2K5AH32S7NS5F6M7GYBF32O5O7QOXN57A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.236599983 (377h51m9.007241318s) [noise=true , address=77.90.31.16:4735 , piece_id=AUOIUCZIJLLNGM37ZWT5DYQWLDXJF363XAMQ2ZFEIAHL6AEAERNQ, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242137387 (377h51m9.012778712s) [noise=true , address=77.90.31.16:4733 , piece_id=LZWLNT7UKJPTJN6CZ5UH6DZ2MYLQE5OTHMQETQKIODJMAPX73IMA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.242995759 (377h51m9.013637094s) [noise=true , address=45.91.116.9:4261 , piece_id=4BSAVF57L56GOJEYLFQBP7CJWVBAP6JMBA7E5MWU5OXZT6R5EXOA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: <nil> Jul 9 03:01:34.243008334 (377h51m9.013649669s) [seg_pos={0 0}, upload=302293]: Segment reached optimal threshold of 13 pieces. Jul 9 03:01:34.243124386 (377h51m9.013765721s) [noise=true , address=45.91.116.8:4047 , piece_id=66XRNA3S672IDAIB6ZIVULIDI23Y24T47OTK6AWASYZJJTZCZUAA, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243179531 (377h51m9.013820866s) [noise=true , address=104.36.82.8:4969 , piece_id=Z5STRISGA5NLFDIDRR2F6DWSYJUQ5X5N7T7MM3ZYMPO4U4SQVR4A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243274663 (377h51m9.013915998s) [noise=true , address=45.91.116.12:4227 , piece_id=2WJA3G36YXWLDNKRZV3HYZCSTTJMZ7PQ4JUK3OWAIBR2IS5OAN5A, seg_pos={0 0}, upload=302293]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 9 03:01:34.243323757 (377h51m9.013965092s) [seg_pos={0 0}, upload=302293]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 03:01:34.243412717 (377h51m9.014054052s) [upload=302293]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 03:01:34.243417376 (377h51m9.014058711s) [upload=302293]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 03:01:34.538397355 (377h51m9.30903869s) [upload=302293]: Done uploading segments... Jul 9 03:01:34.538417173 (377h51m9.309058508s) [upload=302293]: Upload finished. err: <nil> Duration: 5.792139021s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.X..........................................X........] duration: 6.411031605s FAIL (2025-07-09 03:31:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.411053817s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)