Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.........................X..........] duration: 1.896699813s FAIL (2025-08-13 04:59:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.896721245s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 8.517288161s FAIL (2025-08-13 04:58:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.51730864s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X.........................X.........] duration: 1.96590234s FAIL (2025-08-13 04:58:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.965926436s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.849508654s FAIL (2025-08-13 04:57:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.849534262s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 8.796586561s FAIL (2025-08-13 04:57:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.796611339s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.........................X........] duration: 2.181450018s FAIL (2025-08-13 04:57:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.181484051s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.546343685s FAIL (2025-08-13 04:56:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.546367169s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.........................X.......] duration: 1.832069956s FAIL (2025-08-13 04:56:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.832094994s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 8.324318298s FAIL (2025-08-13 04:56:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.324438198s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.........................X......] duration: 1.83223426s FAIL (2025-08-13 04:56:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.832262664s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.967707595s FAIL (2025-08-13 04:56:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.967741029s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 7.951359544s FAIL (2025-08-13 04:55:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.95137824s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X.........................X.....] duration: 1.825747144s FAIL (2025-08-13 04:55:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.825777s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.99597308s FAIL (2025-08-13 04:55:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.995998327s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 7.885466474s FAIL (2025-08-13 04:55:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.885488907s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.914217942s FAIL (2025-08-13 04:54:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.914246035s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.........................X....] duration: 1.918494904s FAIL (2025-08-13 04:54:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.918527825s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.904739895s FAIL (2025-08-13 04:54:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.904767958s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.........................X...] duration: 1.823814718s FAIL (2025-08-13 04:54:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.823853052s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 9.74602359s FAIL (2025-08-13 04:53:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.746050842s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 7.617071507s FAIL (2025-08-13 04:53:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.617114399s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.873077918s FAIL (2025-08-13 04:53:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.873097464s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.........................X..] duration: 1.81206581s FAIL (2025-08-13 04:52:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.812114493s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.884130663s FAIL (2025-08-13 04:52:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.884148368s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X.........................X.] duration: 1.974628675s FAIL (2025-08-13 04:52:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.974655797s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 8.333941256s FAIL (2025-08-13 04:52:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.333964351s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.........................X] duration: 855.342953ms FAIL (2025-08-13 04:52:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 04:52:01.081032033 (107h25m55.512112174s) [upload=111815]: Starting upload Aug 13 04:52:01.081121424 (107h25m55.512201555s) [upload=111815]: Done starting upload Aug 13 04:52:01.081223579 (107h25m55.51230371s) [upload=111815]: Uploading segments... Aug 13 04:52:01.081273565 (107h25m55.512353706s) [upload=111815]: Got next segment. Inline: true Aug 13 04:52:01.081293123 (107h25m55.512373254s) [upload=111815]: Waiting for error group managing segments... Aug 13 04:52:01.081339100 (107h25m55.512419231s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:52:01.081344872 (107h25m55.512425003s) [upload=111815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:52:01.081348118 (107h25m55.512428259s) [upload=111815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:52:01.293748522 (107h25m55.724828673s) [upload=111815]: Done uploading segments... Aug 13 04:52:01.293766095 (107h25m55.724846226s) [upload=111815]: Upload finished. err: metaclient: Too Many Requests Duration:* 855.366598ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 8.223488881s FAIL (2025-08-13 04:51:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.22350881s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.96832617s FAIL (2025-08-13 04:51:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.968343664s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 8.293133899s FAIL (2025-08-13 04:51:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.293184326s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.928871166s FAIL (2025-08-13 04:50:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.928900573s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 8.758403414s FAIL (2025-08-13 04:50:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.758429524s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.678194574s FAIL (2025-08-13 04:50:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.678219494s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 7.862916883s FAIL (2025-08-13 04:50:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.862937282s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.841884358s FAIL (2025-08-13 04:49:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.841911339s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.973744618s FAIL (2025-08-13 04:49:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.973767893s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 7.853647663s FAIL (2025-08-13 04:49:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.853672541s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...................................X.......................] duration: 1.840747803s FAIL (2025-08-13 04:49:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.840774566s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 8.638416309s FAIL (2025-08-13 04:49:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.638773322s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................X................] duration: 5.866965504s FAIL (2025-08-13 04:48:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.866991984s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...................................X......................] duration: 1.839738443s FAIL (2025-08-13 04:48:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.839767768s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...................................X.....................] duration: 1.826459837s FAIL (2025-08-13 04:48:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.826493602s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................................X...............] duration: 5.914156877s FAIL (2025-08-13 04:48:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.914201813s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 8.192539905s FAIL (2025-08-13 04:47:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.19256318s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................................X..............] duration: 6.93109743s FAIL (2025-08-13 04:47:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 6.931120536s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...................................X....................] duration: 1.830454s FAIL (2025-08-13 04:47:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.830499577s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................................X.............] duration: 5.601260583s FAIL (2025-08-13 04:47:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.601300359s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X...................................X...................] duration: 1.860410946s FAIL (2025-08-13 04:47:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c82f41086c10e8e784630232b68e0f7cf143b56088b414b7953f1ff8ef739d6a != before 9bf6ec9994ca7c4766b5318f204568e254175c351cfe104bdd6795e4366e1a5b ======LOGS====== Aug 13 04:31:56.429417659 (107h5m50.860497791s) [upload=111466]: Starting upload Aug 13 04:31:56.429489978 (107h5m50.860570109s) [upload=111466]: Done starting upload Aug 13 04:31:56.429575071 (107h5m50.860655202s) [upload=111466]: Uploading segments... Aug 13 04:31:56.429639935 (107h5m50.860720076s) [upload=111466]: Got next segment. Inline: true Aug 13 04:31:56.429646136 (107h5m50.860726267s) [upload=111466]: Waiting for error group managing segments... Aug 13 04:31:56.429701133 (107h5m50.860781264s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:31:56.429707254 (107h5m50.860787405s) [upload=111466]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 04:31:56.429711382 (107h5m50.860791533s) [upload=111466]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:31:56.721959480 (107h5m51.153039611s) [upload=111466]: Done uploading segments... Aug 13 04:31:56.721978156 (107h5m51.153058277s) [upload=111466]: Upload finished. err: <nil> Duration:* 1.860450231s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 8.194991037s FAIL (2025-08-13 04:46:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.195013289s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................................X............] duration: 5.727004323s FAIL (2025-08-13 04:46:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 04:35:54.793148420 (107h9m49.224228561s) [upload=111532]: Starting upload Aug 13 04:35:54.793209617 (107h9m49.224289748s) [upload=111532]: Done starting upload Aug 13 04:35:54.793316461 (107h9m49.224396593s) [upload=111532]: Uploading segments... Aug 13 04:35:54.793430610 (107h9m49.224510731s) [upload=111532]: Got next segment. Inline: false Aug 13 04:35:54.793554506 (107h9m49.224634637s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 04:35:54.793560358 (107h9m49.224640489s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 04:35:55.251500973 (107h9m49.682581114s) [seg_pos={0 0}, upload=111532]: Begin upload segment... Aug 13 04:35:55.251562933 (107h9m49.682643064s) [seg_pos={0 0}, upload=111532]: Done begin upload segment. Aug 13 04:35:55.251592850 (107h9m49.682672981s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.251640200 (107h9m49.682720331s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252129516 (107h9m49.683209657s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252263212 (107h9m49.683343343s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252446683 (107h9m49.683526814s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252674649 (107h9m49.68375478s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252723483 (107h9m49.683803614s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.252973952 (107h9m49.684054083s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253217899 (107h9m49.68429803s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253493757 (107h9m49.684573888s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253729076 (107h9m49.684809207s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.253951341 (107h9m49.685031473s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254152948 (107h9m49.685233079s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254353331 (107h9m49.685433462s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254560177 (107h9m49.685640308s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254753066 (107h9m49.685833197s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.254998255 (107h9m49.686078387s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:55.255202205 (107h9m49.686282336s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Uploading piece... Aug 13 04:35:56.355719157 (107h9m50.786799288s) [upload=111532]: Next returned nil segment Aug 13 04:35:56.355734617 (107h9m50.786814748s) [upload=111532]: Waiting for error group managing segments... Aug 13 04:35:56.681540013 (107h9m51.112620144s) [noise=true , address=64.49.13.140:4506 , piece_id=Z4QORZQ7AFIAWTKYLX4FMT7627WJA4IEDGCKLSKQHA553E2XISUA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:56.771566774 (107h9m51.202646895s) [noise=true , address=66.23.193.175:4554 , piece_id=EM7YN7DO7KLDVG5KOT3IJN3LEVWPNJOPBNGRBM6GKSPHIS2VEHBA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.205665703 (107h9m51.636745834s) [noise=true , address=104.36.82.9:4025 , piece_id=WPYLP2ZJS6FIWHNNORZJUPIOIQTHIYM6JLLB5LB73Q5SL6IUA55A, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.212355693 (107h9m51.643435824s) [noise=true , address=45.91.116.9:4266 , piece_id=MMFUYW7BHO7D4CPEE2HXIZI2HX6QZX5TIJAFTCU6FOBOCTTWOHUQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.234601727 (107h9m51.665681858s) [noise=true , address=104.36.82.38:4716 , piece_id=5A57PNRXQXOJJEOXWME6GUHK4FMS44LU6VLGJ3ODSQCIXRGWNHYQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.235604687 (107h9m51.666684818s) [noise=true , address=104.36.82.22:4544 , piece_id=3FW6Y7TR2LR2NRTBJIX3KNQAD5KS5ZNQ5IUZLT4GHPOJXY77NZJQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.243559847 (107h9m51.674639998s) [noise=true , address=77.90.31.16:4756 , piece_id=IME22EXQVVOZMGJBVG4C3YKDEKOFQO2MYWXXN7ZWA3IBKWGYM7UQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.245581856 (107h9m51.676661987s) [noise=true , address=104.36.82.23:4458 , piece_id=ML2BS4JJPCWIGWCSMT26VEVHCUTUFEK2WQXLFQLMCYM3JTMVNXEA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.250633902 (107h9m51.681714033s) [noise=true , address=104.36.82.5:4872 , piece_id=BPE5XNM666GKMJFORYPRGAKBK6PQCHCJ6WMEGROMIVOGWDC6KCBQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.312164180 (107h9m51.743244362s) [noise=true , address=104.36.82.61:4394 , piece_id=TCXAXBX3VM4MHJAFMAFMJ6464KAZWZGMRUNOVWOQALJ6J65JTMRA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.325353556 (107h9m51.756433687s) [noise=true , address=104.36.82.35:4594 , piece_id=5XOZS63O2REYEX7O7NNJXN7YHRR72KWMM33AY77BZIGCGH6GZUSQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.345025115 (107h9m51.776105246s) [noise=true , address=77.90.31.19:4850 , piece_id=56VJ72F4HZSPXP2ERIWLI56UGIBFJHTN3TY44ZHMYQ7LU5FKZTSA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.349504736 (107h9m51.780584877s) [noise=true , address=104.36.82.34:4577 , piece_id=MAEYRTTLG4P76ZAOPV6LU63ZWWJMEQDLUZMZ3LYSO5LRPX575QKA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.371997913 (107h9m51.803078064s) [noise=true , address=104.36.82.37:4685 , piece_id=7S2RG7ANIAOTVYUTYMQSJARVIPPOF3EZHJWUOB5XGTEGFVW23YFQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: <nil> Aug 13 04:35:57.372029183 (107h9m51.803109324s) [seg_pos={0 0}, upload=111532]: Segment reached optimal threshold of 14 pieces. Aug 13 04:35:57.372262870 (107h9m51.803343001s) [noise=true , address=104.36.82.44:4884 , piece_id=6ATGNVR6D53RLPJRQZXMAQIUGIGWLCY6SMQYUCQSM7SCVZPCQHEQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372357411 (107h9m51.803437542s) [noise=true , address=79.127.201.215:4360 , piece_id=DLD6VILCBNME5SDDAODLFD34LAIQK3P2JWKNSAU54MAAF62NRYAQ, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372371227 (107h9m51.803451358s) [noise=true , address=104.36.82.59:4336 , piece_id=GTURWRE3L24VECF3IHR5JPJJJHJIXJOCQY6VMRP6EX5OS7I7A3IA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W6uY3xtjFp52F1Q3PwjX8Xb53NPcZxxf3KbxhRafAL6Prm4u5): protocol: expected piece hash; context canceled; EOF Aug 13 04:35:57.372379634 (107h9m51.803459765s) [noise=true , address=104.36.82.60:4354 , piece_id=LV4AO6MUGIWB23DUIRMCPBKDDDAEJQEOGJTWCPXNQPYM6JD5PBGA, seg_pos={0 0}, upload=111532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KTMqMAjUKJ6NFK3tKvnpcKzdSxfnXTBC1hvWce28g3nP3Rhnv): protocol: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54736->104.36.82.60:4354: use of closed network connection Aug 13 04:35:57.372397417 (107h9m51.803477548s) [seg_pos={0 0}, upload=111532]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 04:35:57.372479404 (107h9m51.803559535s) [upload=111532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 04:35:57.372509822 (107h9m51.803589963s) [upload=111532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 04:35:57.681213717 (107h9m52.112293848s) [upload=111532]: Done uploading segments... Aug 13 04:35:57.681234457 (107h9m52.112314588s) [upload=111532]: Upload finished. err: <nil> Duration: 5.727033698s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)