Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [................X.............X...X....X.......X.........X..] duration: 1.878183147s FAIL (2025-08-18 03:30:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:28:22.625299931 (226h2m17.056380062s) [upload=235113]: Starting upload Aug 18 03:28:22.625346430 (226h2m17.056426561s) [upload=235113]: Done starting upload Aug 18 03:28:22.625399371 (226h2m17.056479502s) [upload=235113]: Uploading segments... Aug 18 03:28:22.625434488 (226h2m17.056514619s) [upload=235113]: Got next segment. Inline: true Aug 18 03:28:22.625438285 (226h2m17.056518416s) [upload=235113]: Waiting for error group managing segments... Aug 18 03:28:22.625488291 (226h2m17.056568422s) [upload=235113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:28:22.625492549 (226h2m17.05657267s) [upload=235113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:28:22.625495014 (226h2m17.056575135s) [upload=235113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:28:22.847514783 (226h2m17.278594914s) [upload=235113]: Done uploading segments... Aug 18 03:28:22.847531124 (226h2m17.278611255s) [upload=235113]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.878221761s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.70890254s FAIL (2025-08-18 03:29:55 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.70891758s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X..X..................................X..X...] duration: 5.855197474s FAIL (2025-08-18 03:29:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:26:52.233082115 (226h0m46.664162246s) [upload=235087]: Starting upload Aug 18 03:26:52.233136148 (226h0m46.664216279s) [upload=235087]: Done starting upload Aug 18 03:26:52.233215831 (226h0m46.664295962s) [upload=235087]: Uploading segments... Aug 18 03:26:52.233421034 (226h0m46.664501165s) [upload=235087]: Got next segment. Inline: false Aug 18 03:26:52.233460088 (226h0m46.664540219s) [upload=235087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:26:52.233465107 (226h0m46.664545238s) [upload=235087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:26:52.451725414 (226h0m46.882805545s) [upload=235087]: Next segment err: metaclient: Too Many Requests Aug 18 03:26:52.451757786 (226h0m46.882837917s) [upload=235087]: Done uploading segments... Aug 18 03:26:52.451778295 (226h0m46.882858426s) [upload=235087]: Upload finished. err: metaclient: Too Many Requests Duration: 5.855223192s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.................X.............X...X....X.......X.........X.] duration: 1.892998587s FAIL (2025-08-18 03:29:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:28:22.625299931 (226h2m17.056380062s) [upload=235113]: Starting upload Aug 18 03:28:22.625346430 (226h2m17.056426561s) [upload=235113]: Done starting upload Aug 18 03:28:22.625399371 (226h2m17.056479502s) [upload=235113]: Uploading segments... Aug 18 03:28:22.625434488 (226h2m17.056514619s) [upload=235113]: Got next segment. Inline: true Aug 18 03:28:22.625438285 (226h2m17.056518416s) [upload=235113]: Waiting for error group managing segments... Aug 18 03:28:22.625488291 (226h2m17.056568422s) [upload=235113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:28:22.625492549 (226h2m17.05657267s) [upload=235113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:28:22.625495014 (226h2m17.056575135s) [upload=235113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:28:22.847514783 (226h2m17.278594914s) [upload=235113]: Done uploading segments... Aug 18 03:28:22.847531124 (226h2m17.278611255s) [upload=235113]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.893026831s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 7.426784081s FAIL (2025-08-18 03:28:46 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 7.426804329s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X..X..................................X..X..] duration: 5.863750376s FAIL (2025-08-18 03:28:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:26:52.233082115 (226h0m46.664162246s) [upload=235087]: Starting upload Aug 18 03:26:52.233136148 (226h0m46.664216279s) [upload=235087]: Done starting upload Aug 18 03:26:52.233215831 (226h0m46.664295962s) [upload=235087]: Uploading segments... Aug 18 03:26:52.233421034 (226h0m46.664501165s) [upload=235087]: Got next segment. Inline: false Aug 18 03:26:52.233460088 (226h0m46.664540219s) [upload=235087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:26:52.233465107 (226h0m46.664545238s) [upload=235087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:26:52.451725414 (226h0m46.882805545s) [upload=235087]: Next segment err: metaclient: Too Many Requests Aug 18 03:26:52.451757786 (226h0m46.882837917s) [upload=235087]: Done uploading segments... Aug 18 03:26:52.451778295 (226h0m46.882858426s) [upload=235087]: Upload finished. err: metaclient: Too Many Requests Duration: 5.863776264s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..................X.............X...X....X.......X.........X] duration: 893.945815ms FAIL (2025-08-18 03:28:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:28:22.625299931 (226h2m17.056380062s) [upload=235113]: Starting upload Aug 18 03:28:22.625346430 (226h2m17.056426561s) [upload=235113]: Done starting upload Aug 18 03:28:22.625399371 (226h2m17.056479502s) [upload=235113]: Uploading segments... Aug 18 03:28:22.625434488 (226h2m17.056514619s) [upload=235113]: Got next segment. Inline: true Aug 18 03:28:22.625438285 (226h2m17.056518416s) [upload=235113]: Waiting for error group managing segments... Aug 18 03:28:22.625488291 (226h2m17.056568422s) [upload=235113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:28:22.625492549 (226h2m17.05657267s) [upload=235113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:28:22.625495014 (226h2m17.056575135s) [upload=235113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:28:22.847514783 (226h2m17.278594914s) [upload=235113]: Done uploading segments... Aug 18 03:28:22.847531124 (226h2m17.278611255s) [upload=235113]: Upload finished. err: metaclient: Too Many Requests Duration:* 893.97456ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...................X.............X...X....X.......X.........] duration: 2.021857632s FAIL (2025-08-18 03:28:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 2.021889963s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.66823274s FAIL (2025-08-18 03:27:49 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.668329156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....................X.............X...X....X.......X........] duration: 2.088089239s FAIL (2025-08-18 03:27:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 2.08811564s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X..X..................................X..X.] duration: 6.042076674s FAIL (2025-08-18 03:27:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:26:52.233082115 (226h0m46.664162246s) [upload=235087]: Starting upload Aug 18 03:26:52.233136148 (226h0m46.664216279s) [upload=235087]: Done starting upload Aug 18 03:26:52.233215831 (226h0m46.664295962s) [upload=235087]: Uploading segments... Aug 18 03:26:52.233421034 (226h0m46.664501165s) [upload=235087]: Got next segment. Inline: false Aug 18 03:26:52.233460088 (226h0m46.664540219s) [upload=235087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:26:52.233465107 (226h0m46.664545238s) [upload=235087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:26:52.451725414 (226h0m46.882805545s) [upload=235087]: Next segment err: metaclient: Too Many Requests Aug 18 03:26:52.451757786 (226h0m46.882837917s) [upload=235087]: Done uploading segments... Aug 18 03:26:52.451778295 (226h0m46.882858426s) [upload=235087]: Upload finished. err: metaclient: Too Many Requests Duration: 6.042098185s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.633211169s FAIL (2025-08-18 03:27:15 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.633227108s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X..X..................................X..X] duration: 879.781732ms FAIL (2025-08-18 03:26:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:26:52.233082115 (226h0m46.664162246s) [upload=235087]: Starting upload Aug 18 03:26:52.233136148 (226h0m46.664216279s) [upload=235087]: Done starting upload Aug 18 03:26:52.233215831 (226h0m46.664295962s) [upload=235087]: Uploading segments... Aug 18 03:26:52.233421034 (226h0m46.664501165s) [upload=235087]: Got next segment. Inline: false Aug 18 03:26:52.233460088 (226h0m46.664540219s) [upload=235087]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:26:52.233465107 (226h0m46.664545238s) [upload=235087]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:26:52.451725414 (226h0m46.882805545s) [upload=235087]: Next segment err: metaclient: Too Many Requests Aug 18 03:26:52.451757786 (226h0m46.882837917s) [upload=235087]: Done uploading segments... Aug 18 03:26:52.451778295 (226h0m46.882858426s) [upload=235087]: Upload finished. err: metaclient: Too Many Requests Duration: 879.80782ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....................X.............X...X....X.......X.......] duration: 2.04716698s FAIL (2025-08-18 03:26:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 2.047194622s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.803174286s FAIL (2025-08-18 03:26:04 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.803203833s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X..X..................................X..] duration: 5.658083365s FAIL (2025-08-18 03:25:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:24:53.065879381 (225h58m47.496959512s) [upload=235055]: Starting upload Aug 18 03:24:53.065927853 (225h58m47.497007984s) [upload=235055]: Done starting upload Aug 18 03:24:53.066036330 (225h58m47.497116451s) [upload=235055]: Uploading segments... Aug 18 03:24:53.066232917 (225h58m47.497313038s) [upload=235055]: Got next segment. Inline: false Aug 18 03:24:53.066268445 (225h58m47.497348576s) [upload=235055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:24:53.066273504 (225h58m47.497353625s) [upload=235055]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:24:53.285383948 (225h58m47.716464069s) [upload=235055]: Next segment err: metaclient: Too Many Requests Aug 18 03:24:53.285410238 (225h58m47.716490359s) [upload=235055]: Done uploading segments... Aug 18 03:24:53.285420688 (225h58m47.716500819s) [upload=235055]: Upload finished. err: metaclient: Too Many Requests Duration: 5.658108683s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [......................X.............X...X....X.......X......] duration: 1.916816279s FAIL (2025-08-18 03:25:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.916843783s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.440100944s FAIL (2025-08-18 03:25:25 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.440171909s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X..X..................................X.] duration: 5.694732268s FAIL (2025-08-18 03:25:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:24:53.065879381 (225h58m47.496959512s) [upload=235055]: Starting upload Aug 18 03:24:53.065927853 (225h58m47.497007984s) [upload=235055]: Done starting upload Aug 18 03:24:53.066036330 (225h58m47.497116451s) [upload=235055]: Uploading segments... Aug 18 03:24:53.066232917 (225h58m47.497313038s) [upload=235055]: Got next segment. Inline: false Aug 18 03:24:53.066268445 (225h58m47.497348576s) [upload=235055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:24:53.066273504 (225h58m47.497353625s) [upload=235055]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:24:53.285383948 (225h58m47.716464069s) [upload=235055]: Next segment err: metaclient: Too Many Requests Aug 18 03:24:53.285410238 (225h58m47.716490359s) [upload=235055]: Done uploading segments... Aug 18 03:24:53.285420688 (225h58m47.716500819s) [upload=235055]: Upload finished. err: metaclient: Too Many Requests Duration: 5.69475951s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.503541032s FAIL (2025-08-18 03:25:01 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.503565999s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X..X..................................X] duration: 883.297809ms FAIL (2025-08-18 03:24:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:24:53.065879381 (225h58m47.496959512s) [upload=235055]: Starting upload Aug 18 03:24:53.065927853 (225h58m47.497007984s) [upload=235055]: Done starting upload Aug 18 03:24:53.066036330 (225h58m47.497116451s) [upload=235055]: Uploading segments... Aug 18 03:24:53.066232917 (225h58m47.497313038s) [upload=235055]: Got next segment. Inline: false Aug 18 03:24:53.066268445 (225h58m47.497348576s) [upload=235055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:24:53.066273504 (225h58m47.497353625s) [upload=235055]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 03:24:53.285383948 (225h58m47.716464069s) [upload=235055]: Next segment err: metaclient: Too Many Requests Aug 18 03:24:53.285410238 (225h58m47.716490359s) [upload=235055]: Done uploading segments... Aug 18 03:24:53.285420688 (225h58m47.716500819s) [upload=235055]: Upload finished. err: metaclient: Too Many Requests Duration: 883.31913ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.669678994s FAIL (2025-08-18 03:24:43 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.66970277s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......................X.............X...X....X.......X.....] duration: 1.888698608s FAIL (2025-08-18 03:24:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.888726451s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.00042092s FAIL (2025-08-18 03:24:23 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 7.000442021s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X..X..................................] duration: 5.94490982s FAIL (2025-08-18 03:24:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.944929048s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [........................X.............X...X....X.......X....] duration: 1.892616744s FAIL (2025-08-18 03:24:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.892646862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.........................X.............X...X....X.......X...] duration: 1.861926103s FAIL (2025-08-18 03:23:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.86196146s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.537414162s FAIL (2025-08-18 03:23:27 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.537436747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X..X.................................] duration: 5.902511547s FAIL (2025-08-18 03:23:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.9025394s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.852878798s FAIL (2025-08-18 03:23:02 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.852899176s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..........................X.............X...X....X.......X..] duration: 1.954801268s FAIL (2025-08-18 03:22:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.954827398s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X..X................................] duration: 5.903814913s FAIL (2025-08-18 03:21:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.90383839s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.575901499s FAIL (2025-08-18 03:21:51 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.575924643s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...........................X.............X...X....X.......X.] duration: 1.848986925s FAIL (2025-08-18 03:21:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.849013768s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X..X...............................] duration: 5.791627149s FAIL (2025-08-18 03:21:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.791651577s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [............................X.............X...X....X.......X] duration: 1.441004278s FAIL (2025-08-18 03:21:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 18 03:21:04.347791815 (225h54m58.778871946s) [upload=234993]: Starting upload Aug 18 03:21:04.347838826 (225h54m58.778918957s) [upload=234993]: Done starting upload Aug 18 03:21:04.347885645 (225h54m58.778965777s) [upload=234993]: Uploading segments... Aug 18 03:21:04.347938185 (225h54m58.779018316s) [upload=234993]: Got next segment. Inline: true Aug 18 03:21:04.347943326 (225h54m58.779023457s) [upload=234993]: Waiting for error group managing segments... Aug 18 03:21:04.347977892 (225h54m58.779058023s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:21:04.347982160 (225h54m58.779062291s) [upload=234993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:21:04.347984544 (225h54m58.779064665s) [upload=234993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:21:04.646580744 (225h54m59.077660875s) [upload=234993]: Done uploading segments... Aug 18 03:21:04.646598989 (225h54m59.07767912s) [upload=234993]: Upload finished. err: <nil> Duration:* 1.441033033s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.656893211s FAIL (2025-08-18 03:20:42 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.656913429s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X..X..............................] duration: 5.725643223s FAIL (2025-08-18 03:20:38 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.725677339s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.505883732s FAIL (2025-08-18 03:20:13 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.505899883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............................X.............X...X....X.......] duration: 2.118117977s FAIL (2025-08-18 03:20:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:14:00.235887345 (225h47m54.666967466s) [upload=234864]: Starting upload Aug 18 03:14:00.235937891 (225h47m54.667018023s) [upload=234864]: Done starting upload Aug 18 03:14:00.235989049 (225h47m54.667069171s) [upload=234864]: Uploading segments... Aug 18 03:14:00.236022383 (225h47m54.667102514s) [upload=234864]: Got next segment. Inline: true Aug 18 03:14:00.236026120 (225h47m54.667106241s) [upload=234864]: Waiting for error group managing segments... Aug 18 03:14:00.236059505 (225h47m54.667139626s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:14:00.236063362 (225h47m54.667143493s) [upload=234864]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:14:00.236065967 (225h47m54.667146098s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:14:00.530460418 (225h47m54.961540549s) [upload=234864]: Done uploading segments... Aug 18 03:14:00.530475097 (225h47m54.961555228s) [upload=234864]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.118157773s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.549661488s FAIL (2025-08-18 03:19:32 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.549681306s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X..X.............................] duration: 5.856698442s FAIL (2025-08-18 03:19:30 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.856724844s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 7.467463877s FAIL (2025-08-18 03:18:58 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 7.467487203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............................X.............X...X....X......] duration: 2.063064349s FAIL (2025-08-18 03:18:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:14:00.235887345 (225h47m54.666967466s) [upload=234864]: Starting upload Aug 18 03:14:00.235937891 (225h47m54.667018023s) [upload=234864]: Done starting upload Aug 18 03:14:00.235989049 (225h47m54.667069171s) [upload=234864]: Uploading segments... Aug 18 03:14:00.236022383 (225h47m54.667102514s) [upload=234864]: Got next segment. Inline: true Aug 18 03:14:00.236026120 (225h47m54.667106241s) [upload=234864]: Waiting for error group managing segments... Aug 18 03:14:00.236059505 (225h47m54.667139626s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:14:00.236063362 (225h47m54.667143493s) [upload=234864]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:14:00.236065967 (225h47m54.667146098s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:14:00.530460418 (225h47m54.961540549s) [upload=234864]: Done uploading segments... Aug 18 03:14:00.530475097 (225h47m54.961555228s) [upload=234864]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.063102894s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X..X............................] duration: 5.686891105s FAIL (2025-08-18 03:18:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.686913307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.343506923s FAIL (2025-08-18 03:18:07 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.343527952s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............................X.............X...X....X.....] duration: 1.936336377s FAIL (2025-08-18 03:17:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:14:00.235887345 (225h47m54.666967466s) [upload=234864]: Starting upload Aug 18 03:14:00.235937891 (225h47m54.667018023s) [upload=234864]: Done starting upload Aug 18 03:14:00.235989049 (225h47m54.667069171s) [upload=234864]: Uploading segments... Aug 18 03:14:00.236022383 (225h47m54.667102514s) [upload=234864]: Got next segment. Inline: true Aug 18 03:14:00.236026120 (225h47m54.667106241s) [upload=234864]: Waiting for error group managing segments... Aug 18 03:14:00.236059505 (225h47m54.667139626s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:14:00.236063362 (225h47m54.667143493s) [upload=234864]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:14:00.236065967 (225h47m54.667146098s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:14:00.530460418 (225h47m54.961540549s) [upload=234864]: Done uploading segments... Aug 18 03:14:00.530475097 (225h47m54.961555228s) [upload=234864]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.936361095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X..X...........................] duration: 5.823486195s FAIL (2025-08-18 03:17:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 18 02:57:44.076573606 (225h31m38.507653737s) [upload=234575]: Starting upload Aug 18 02:57:44.076612601 (225h31m38.507692732s) [upload=234575]: Done starting upload Aug 18 02:57:44.076788968 (225h31m38.507869099s) [upload=234575]: Uploading segments... Aug 18 02:57:44.076826861 (225h31m38.507906992s) [upload=234575]: Got next segment. Inline: false Aug 18 02:57:44.076861867 (225h31m38.507941999s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 02:57:44.076865795 (225h31m38.507945916s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 18 02:57:44.537523372 (225h31m38.968603503s) [seg_pos={0 0}, upload=234575]: Begin upload segment... Aug 18 02:57:44.537572927 (225h31m38.968653058s) [seg_pos={0 0}, upload=234575]: Done begin upload segment. Aug 18 02:57:44.537596112 (225h31m38.968676243s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537672077 (225h31m38.968752218s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.537986328 (225h31m38.969066459s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538268438 (225h31m38.969348569s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538352789 (225h31m38.96943292s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538531652 (225h31m38.969611783s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538780518 (225h31m38.969860639s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.538816527 (225h31m38.969896658s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539032269 (225h31m38.970112401s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539237842 (225h31m38.970317973s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539281035 (225h31m38.970361166s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539529601 (225h31m38.970609732s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539624894 (225h31m38.970705025s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539723332 (225h31m38.970803463s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539925308 (225h31m38.971005439s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.539999871 (225h31m38.971080002s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.541980039 (225h31m38.97306017s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:44.542324960 (225h31m38.973405091s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Uploading piece... Aug 18 02:57:46.047782220 (225h31m40.478862351s) [upload=234575]: Next returned nil segment Aug 18 02:57:46.047803701 (225h31m40.478883822s) [upload=234575]: Waiting for error group managing segments... Aug 18 02:57:46.439201889 (225h31m40.87028202s) [noise=true , address=77.90.31.10:4531 , piece_id=TR5EBV4J7P23W4TJJXC3JJWLIJ3HBSYHNFUW53GZHAXWAT5NOZ5Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.445037504 (225h31m40.876117645s) [noise=true , address=77.90.31.11:4559 , piece_id=J2TXGSYILZ4NSHG2TBX54CEYNRH5TGFT4TRSUMVNMUL4TK2IUN6Q, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.465103759 (225h31m40.89618389s) [noise=true , address=104.36.82.23:4458 , piece_id=HV4A7BXTSEJYRPLT6KPANQBQEGJKN3DVZVJHQ3FH72OWF2X54PSA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.503798558 (225h31m40.934878689s) [noise=true , address=77.90.31.18:4812 , piece_id=JGRQ4LJIUP5PZM4BOQNFKJHYFXP5RMAPPRESC3IYG54MZPEBCAGQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.512519184 (225h31m40.943599315s) [noise=true , address=104.36.82.13:4215 , piece_id=CVDXKFZVO4BJFC2UVUGROXXUXF3T5EGRB7MYCAVQNCUKHG7CEZHQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.517720015 (225h31m40.948800146s) [noise=true , address=104.36.82.44:4873 , piece_id=C3QBAMXSFEIDYDIEGDGHNDEAJVZYDEDCOI32TDQY5HG2FA2UAFVA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520008413 (225h31m40.951088544s) [noise=true , address=104.36.82.36:4627 , piece_id=5STJ37EAL6F3YRL5TMKAJ47HCXORB57A7PIOVTYZL3RDWOJGD6BQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.520813864 (225h31m40.951893995s) [noise=true , address=77.90.31.19:4870 , piece_id=3KLQLMBFPPJBLE2PASHPBRIRL637M3TYRQYGZ36KR3H7NGPVICNQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.547402960 (225h31m40.978483091s) [noise=true , address=104.36.82.22:4523 , piece_id=2GXY4FXKS7GHZYND674VC2BEACZCIA7W254F477KGPLJIM366EJA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.556688566 (225h31m40.987768697s) [noise=true , address=104.36.82.38:4710 , piece_id=VI3IWQWYBUGUMUTAJWUYJJN2CTVHKFZEH3EK7B73FMQPQ7GZX6JA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.563136424 (225h31m40.994216555s) [noise=true , address=104.36.82.62:4435 , piece_id=XBTGVTMUOUBAHCFFIFS3N5CI7NNVDDRX54DA6JWOLDDGMFK6WMBQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.573269292 (225h31m41.004349423s) [noise=true , address=77.90.31.16:4736 , piece_id=WFZ6BC4YFOKAK2FGD6UUX22CDMRYC5EBDGNGOXFHKRHXTGGVOZOA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.576313607 (225h31m41.007393778s) [noise=true , address=45.91.116.10:4086 , piece_id=PNJZM7SSI7WHT6KPRMYSFYDBDUBMH63CA2ER7H7CAXFDPNWOHORA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592627222 (225h31m41.023707353s) [noise=true , address=104.36.82.58:4273 , piece_id=M3GZ3KVHQRPL2SEE6KRHZIIPF6Q2TOP2UL4PDWCN3KORMMG3SJQA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: <nil> Aug 18 02:57:46.592654254 (225h31m41.023734385s) [seg_pos={0 0}, upload=234575]: Segment reached optimal threshold of 14 pieces. Aug 18 02:57:46.592798780 (225h31m41.023878911s) [noise=true , address=104.36.82.10:4092 , piece_id=7KIIZIUKOZ36K5OBA2EGEPBUDFRSNCIH4HT6E2DKSBBCF7WYR3EA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592855819 (225h31m41.02393595s) [noise=true , address=104.36.82.45:4930 , piece_id=3ATHJW4MHXTRNG6FYH2WX2ESJED3ZV6FJERN4J3PCLJ2Z3F5MUHA, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qaYFSpezuLAc5zJfAYruzad6nTpYFDpog7bmKMVTA8MVm76NB): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592897157 (225h31m41.023977289s) [noise=true , address=104.36.82.34:4565 , piece_id=JSBZVDSB4VPIOHSESIWJQR7C6WQ2AJ63SI3LA62TJICLOXGVJ5QQ, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JRxcRCxwuNJrRRcKKxEggFpmRk2fo9kdVbmhPn3eLLeiQop9R): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592943997 (225h31m41.024024128s) [noise=true , address=104.36.82.49:4002 , piece_id=LSHMG4HEJVONPTOLILC5IBJF337LTHM2AC6LOSSWFHU66A6Y545A, seg_pos={0 0}, upload=234575]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: expected piece hash; context canceled; EOF Aug 18 02:57:46.592961581 (225h31m41.024041702s) [seg_pos={0 0}, upload=234575]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 18 02:57:46.593047906 (225h31m41.024128037s) [upload=234575]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 18 02:57:46.593052365 (225h31m41.024132496s) [upload=234575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 02:57:46.900968097 (225h31m41.332048228s) [upload=234575]: Done uploading segments... Aug 18 02:57:46.900986613 (225h31m41.332066734s) [upload=234575]: Upload finished. err: <nil> Duration: 5.823502876s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [................................X.............X...X....X....] duration: 1.892828253s FAIL (2025-08-18 03:17:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 03:14:00.235887345 (225h47m54.666967466s) [upload=234864]: Starting upload Aug 18 03:14:00.235937891 (225h47m54.667018023s) [upload=234864]: Done starting upload Aug 18 03:14:00.235989049 (225h47m54.667069171s) [upload=234864]: Uploading segments... Aug 18 03:14:00.236022383 (225h47m54.667102514s) [upload=234864]: Got next segment. Inline: true Aug 18 03:14:00.236026120 (225h47m54.667106241s) [upload=234864]: Waiting for error group managing segments... Aug 18 03:14:00.236059505 (225h47m54.667139626s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 18 03:14:00.236063362 (225h47m54.667143493s) [upload=234864]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 18 03:14:00.236065967 (225h47m54.667146098s) [upload=234864]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 18 03:14:00.530460418 (225h47m54.961540549s) [upload=234864]: Done uploading segments... Aug 18 03:14:00.530475097 (225h47m54.961555228s) [upload=234864]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.892855233s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.904546131s FAIL (2025-08-18 03:17:19 UTC) orbiter warning: read tcp 188.166.225.170:47770->136.0.77.2:443: read: connection reset by peer Duration: 6.904568303s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)