Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...............................X..X.............X.....X.X...] duration: 1.967433328s FAIL (2025-08-13 14:27:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 14:25:57.508778246 (116h59m51.939858377s) [upload=121717]: Starting upload Aug 13 14:25:57.508850885 (116h59m51.939931016s) [upload=121717]: Done starting upload Aug 13 14:25:57.508940917 (116h59m51.940021058s) [upload=121717]: Uploading segments... Aug 13 14:25:57.509065897 (116h59m51.940146038s) [upload=121717]: Got next segment. Inline: true Aug 13 14:25:57.509071818 (116h59m51.940151959s) [upload=121717]: Waiting for error group managing segments... Aug 13 14:25:57.509160628 (116h59m51.940240769s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:57.509169355 (116h59m51.940249496s) [upload=121717]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:25:57.509174675 (116h59m51.940254806s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:58.324969756 (116h59m52.756049887s) [upload=121717]: Done uploading segments... Aug 13 14:25:58.324989163 (116h59m52.756069294s) [upload=121717]: Upload finished. err: <nil> Duration:* 1.967460911s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X.......X.X.................] duration: 5.904179557s FAIL (2025-08-13 14:27:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.904202761s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [................................X..X.............X.....X.X..] duration: 1.971035738s FAIL (2025-08-13 14:27:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 14:25:57.508778246 (116h59m51.939858377s) [upload=121717]: Starting upload Aug 13 14:25:57.508850885 (116h59m51.939931016s) [upload=121717]: Done starting upload Aug 13 14:25:57.508940917 (116h59m51.940021058s) [upload=121717]: Uploading segments... Aug 13 14:25:57.509065897 (116h59m51.940146038s) [upload=121717]: Got next segment. Inline: true Aug 13 14:25:57.509071818 (116h59m51.940151959s) [upload=121717]: Waiting for error group managing segments... Aug 13 14:25:57.509160628 (116h59m51.940240769s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:57.509169355 (116h59m51.940249496s) [upload=121717]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:25:57.509174675 (116h59m51.940254806s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:58.324969756 (116h59m52.756049887s) [upload=121717]: Done uploading segments... Aug 13 14:25:58.324989163 (116h59m52.756069294s) [upload=121717]: Upload finished. err: <nil> Duration:* 1.971071587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X..................X.....X......X.....................X..] duration: 6.513128876s FAIL (2025-08-13 14:27:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 14:25:34.670450049 (116h59m29.10153018s) [upload=121710]: Starting upload Aug 13 14:25:34.670500074 (116h59m29.101580205s) [upload=121710]: Done starting upload Aug 13 14:25:34.670554288 (116h59m29.101634409s) [upload=121710]: Uploading segments... Aug 13 14:25:34.670715417 (116h59m29.101795548s) [upload=121710]: Got next segment. Inline: false Aug 13 14:25:34.670762726 (116h59m29.101842857s) [upload=121710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:34.670769440 (116h59m29.101849631s) [upload=121710]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:25:35.141817037 (116h59m29.572897168s) [seg_pos={0 0}, upload=121710]: Begin upload segment... Aug 13 14:25:35.141863436 (116h59m29.572943567s) [seg_pos={0 0}, upload=121710]: Done begin upload segment. Aug 13 14:25:35.141887462 (116h59m29.572967593s) [noise=true , address=104.36.82.38:4719 , piece_id=PYZMWL3OZIULLOGDVKRF2RSLGLDGGWTCFWHQ66CPKPPEP6JGCVXQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.141902260 (116h59m29.572982391s) [noise=true , address=104.36.82.59:4328 , piece_id=EVWPCNQWESLH26NS62FKFXKOIW64GUDZFCABBK6H4FSYP35OA5OA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.141945994 (116h59m29.573026125s) [noise=true , address=104.36.82.56:4235 , piece_id=HS4RRPCLDB5ZMD2EJZZCTSCWVY3TWZC6QDVS4TRXRHO3EGZGR7UQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142200952 (116h59m29.573281083s) [noise=true , address=104.36.82.60:4371 , piece_id=RTCVTBKO26FY7ZUNFYYJ2IOQ5XJCGG774ZP4WFLOVK5D47ULGXOA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142305862 (116h59m29.573385993s) [noise=true , address=104.36.82.44:4874 , piece_id=C2ATKCPCO7P4KSHSMRRNGYCMH4IRDKUHL5ERDJKIJ4OECUCLQCFQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142439999 (116h59m29.57352013s) [noise=true , address=104.36.82.19:4408 , piece_id=LT43HOEP3H4NTUKHSRUXPSRO4XBEJ6NOHIAPCBPZKVXPIW7SLLQQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142605826 (116h59m29.573685957s) [noise=true , address=77.90.31.15:4702 , piece_id=SOXOEHUIJGYJKQ64YWBZMGAZMQJHR5RQMP3VFS5SGDSFM6J52ZMQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142682213 (116h59m29.573762344s) [noise=true , address=77.90.31.19:4846 , piece_id=3UYXBMMHS5MPUNXI6PY65K3AGHIIIVSGXJDEUVXLTVXCZYWTL7JA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142910119 (116h59m29.57399024s) [noise=true , address=104.36.82.37:4678 , piece_id=S5AV5WE4DOUZIRLT3GMKQLPLHARLF24HTVG3QNJ2GWZPYIC55PGQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142917733 (116h59m29.573997864s) [noise=true , address=66.23.193.175:4564 , piece_id=S4CHPP736M3D5GOURHJRONGXSYNTREBY3S243VJ6E5ZM75TDTLPA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143203881 (116h59m29.574284012s) [noise=true , address=64.49.13.140:4511 , piece_id=VTJ5NDREGII5G4IOP44JNYRJ5IFHHRP7S7OWTRFOGL4FCKBAUDYA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143239018 (116h59m29.574319149s) [noise=true , address=104.36.82.34:4560 , piece_id=4ZZFZKJRUXPMKOE2RDH6ZWFP3TVUOPSRQS7FAB2YANQ3MKA7JLWQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143481342 (116h59m29.574561473s) [noise=true , address=104.36.82.39:4727 , piece_id=UYFASTBHHBMDAY7ZUYHNSNSDDZMPPZ4AIR6X6HC6ERBXLXF4AOUQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143486071 (116h59m29.574566192s) [noise=true , address=104.36.82.23:4474 , piece_id=MDCBWJSKMP57ZE2V5WIKCG7QWIMRAOOIIFAAJQ5YXVWBEG2EG45Q, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143749975 (116h59m29.574830106s) [noise=true , address=77.90.31.11:4580 , piece_id=CBVCJWDPVPPQW2JIAV25UUFT7EVH62WJOP3FK4M7GUVJAO6P2QGA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.144014762 (116h59m29.575094893s) [noise=true , address=104.36.82.62:4412 , piece_id=CTYSBOF4JVZB5JTYNQDMP6Q5XK5KVETQH5SHHSR65K6HF3TZM2WQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.146502863 (116h59m29.577582994s) [noise=true , address=79.127.201.215:4358 , piece_id=5LNV5OH2YBCRCO7GJOK6Q3VSYC47ZJ4VTZQO3XVN7BLIRFRB4J5A, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.146602894 (116h59m29.577683025s) [noise=true , address=104.36.82.36:4642 , piece_id=TILD5YD3ZU4AOT5227X6PIEIKYXF52D2AZSNDWLOENLTQ4ZMVSLA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:36.312798371 (116h59m30.743878492s) [upload=121710]: Next returned nil segment Aug 13 14:25:36.312814072 (116h59m30.743894203s) [upload=121710]: Waiting for error group managing segments... Aug 13 14:25:36.608833520 (116h59m31.039913651s) [noise=true , address=66.23.193.175:4564 , piece_id=S4CHPP736M3D5GOURHJRONGXSYNTREBY3S243VJ6E5ZM75TDTLPA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.109920204 (116h59m31.541000325s) [noise=true , address=79.127.201.215:4358 , piece_id=5LNV5OH2YBCRCO7GJOK6Q3VSYC47ZJ4VTZQO3XVN7BLIRFRB4J5A, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.110478522 (116h59m31.541558653s) [noise=true , address=104.36.82.36:4642 , piece_id=TILD5YD3ZU4AOT5227X6PIEIKYXF52D2AZSNDWLOENLTQ4ZMVSLA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.127900888 (116h59m31.558981009s) [noise=true , address=77.90.31.11:4580 , piece_id=CBVCJWDPVPPQW2JIAV25UUFT7EVH62WJOP3FK4M7GUVJAO6P2QGA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.166070970 (116h59m31.597151101s) [noise=true , address=104.36.82.37:4678 , piece_id=S5AV5WE4DOUZIRLT3GMKQLPLHARLF24HTVG3QNJ2GWZPYIC55PGQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.167794408 (116h59m31.598874539s) [noise=true , address=104.36.82.19:4408 , piece_id=LT43HOEP3H4NTUKHSRUXPSRO4XBEJ6NOHIAPCBPZKVXPIW7SLLQQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.167973721 (116h59m31.599053852s) [noise=true , address=104.36.82.59:4328 , piece_id=EVWPCNQWESLH26NS62FKFXKOIW64GUDZFCABBK6H4FSYP35OA5OA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.195711503 (116h59m31.626791634s) [noise=true , address=104.36.82.34:4560 , piece_id=4ZZFZKJRUXPMKOE2RDH6ZWFP3TVUOPSRQS7FAB2YANQ3MKA7JLWQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.209376008 (116h59m31.640456129s) [noise=true , address=104.36.82.39:4727 , piece_id=UYFASTBHHBMDAY7ZUYHNSNSDDZMPPZ4AIR6X6HC6ERBXLXF4AOUQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.252511451 (116h59m31.683591582s) [noise=true , address=77.90.31.19:4846 , piece_id=3UYXBMMHS5MPUNXI6PY65K3AGHIIIVSGXJDEUVXLTVXCZYWTL7JA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.256659508 (116h59m31.687739639s) [noise=true , address=104.36.82.38:4719 , piece_id=PYZMWL3OZIULLOGDVKRF2RSLGLDGGWTCFWHQ66CPKPPEP6JGCVXQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.294487646 (116h59m31.725567777s) [noise=true , address=77.90.31.15:4702 , piece_id=SOXOEHUIJGYJKQ64YWBZMGAZMQJHR5RQMP3VFS5SGDSFM6J52ZMQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.310182075 (116h59m31.741262206s) [noise=true , address=104.36.82.62:4412 , piece_id=CTYSBOF4JVZB5JTYNQDMP6Q5XK5KVETQH5SHHSR65K6HF3TZM2WQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.514693066 (116h59m31.945773197s) [noise=true , address=104.36.82.23:4474 , piece_id=MDCBWJSKMP57ZE2V5WIKCG7QWIMRAOOIIFAAJQ5YXVWBEG2EG45Q, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.514716501 (116h59m31.945796632s) [seg_pos={0 0}, upload=121710]: Segment reached optimal threshold of 14 pieces. Aug 13 14:25:37.514831381 (116h59m31.945911512s) [noise=true , address=64.49.13.140:4511 , piece_id=VTJ5NDREGII5G4IOP44JNYRJ5IFHHRP7S7OWTRFOGL4FCKBAUDYA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: expected piece hash; context canceled; EOF Aug 13 14:25:37.514874733 (116h59m31.945954864s) [noise=true , address=104.36.82.56:4235 , piece_id=HS4RRPCLDB5ZMD2EJZZCTSCWVY3TWZC6QDVS4TRXRHO3EGZGR7UQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1a1sW1G1WPL5WQ2XxprWKQMsJkcb5ETSkk3ULp1TjRaZWptaT): protocol: expected piece hash; context canceled; EOF Aug 13 14:25:37.515019360 (116h59m31.946099492s) [noise=true , address=104.36.82.44:4874 , piece_id=C2ATKCPCO7P4KSHSMRRNGYCMH4IRDKUHL5ERDJKIJ4OECUCLQCFQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection; write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection Aug 13 14:25:37.515053225 (116h59m31.946133356s) [noise=true , address=104.36.82.60:4371 , piece_id=RTCVTBKO26FY7ZUNFYYJ2IOQ5XJCGG774ZP4WFLOVK5D47ULGXOA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KJP5DHmfnSFQF3Q74GzMUvcAz9zh16td4pnsqHAwToiM4D1vd): protocol: write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection; write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection Aug 13 14:25:37.515074867 (116h59m31.946154998s) [seg_pos={0 0}, upload=121710]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:25:37.515134190 (116h59m31.946214321s) [upload=121710]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:25:37.515138007 (116h59m31.946218138s) [upload=121710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:38.573880469 (116h59m33.0049606s) [upload=121710]: Done uploading segments... Aug 13 14:25:38.573909565 (116h59m33.004989686s) [upload=121710]: Upload finished. err: <nil> Duration: 6.513152271s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................X.......X.X................] duration: 5.952742927s FAIL (2025-08-13 14:26:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.952764998s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X..................X.....X......X.....................X.] duration: 6.290634748s FAIL (2025-08-13 14:26:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 14:25:34.670450049 (116h59m29.10153018s) [upload=121710]: Starting upload Aug 13 14:25:34.670500074 (116h59m29.101580205s) [upload=121710]: Done starting upload Aug 13 14:25:34.670554288 (116h59m29.101634409s) [upload=121710]: Uploading segments... Aug 13 14:25:34.670715417 (116h59m29.101795548s) [upload=121710]: Got next segment. Inline: false Aug 13 14:25:34.670762726 (116h59m29.101842857s) [upload=121710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:34.670769440 (116h59m29.101849631s) [upload=121710]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:25:35.141817037 (116h59m29.572897168s) [seg_pos={0 0}, upload=121710]: Begin upload segment... Aug 13 14:25:35.141863436 (116h59m29.572943567s) [seg_pos={0 0}, upload=121710]: Done begin upload segment. Aug 13 14:25:35.141887462 (116h59m29.572967593s) [noise=true , address=104.36.82.38:4719 , piece_id=PYZMWL3OZIULLOGDVKRF2RSLGLDGGWTCFWHQ66CPKPPEP6JGCVXQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.141902260 (116h59m29.572982391s) [noise=true , address=104.36.82.59:4328 , piece_id=EVWPCNQWESLH26NS62FKFXKOIW64GUDZFCABBK6H4FSYP35OA5OA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.141945994 (116h59m29.573026125s) [noise=true , address=104.36.82.56:4235 , piece_id=HS4RRPCLDB5ZMD2EJZZCTSCWVY3TWZC6QDVS4TRXRHO3EGZGR7UQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142200952 (116h59m29.573281083s) [noise=true , address=104.36.82.60:4371 , piece_id=RTCVTBKO26FY7ZUNFYYJ2IOQ5XJCGG774ZP4WFLOVK5D47ULGXOA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142305862 (116h59m29.573385993s) [noise=true , address=104.36.82.44:4874 , piece_id=C2ATKCPCO7P4KSHSMRRNGYCMH4IRDKUHL5ERDJKIJ4OECUCLQCFQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142439999 (116h59m29.57352013s) [noise=true , address=104.36.82.19:4408 , piece_id=LT43HOEP3H4NTUKHSRUXPSRO4XBEJ6NOHIAPCBPZKVXPIW7SLLQQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142605826 (116h59m29.573685957s) [noise=true , address=77.90.31.15:4702 , piece_id=SOXOEHUIJGYJKQ64YWBZMGAZMQJHR5RQMP3VFS5SGDSFM6J52ZMQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142682213 (116h59m29.573762344s) [noise=true , address=77.90.31.19:4846 , piece_id=3UYXBMMHS5MPUNXI6PY65K3AGHIIIVSGXJDEUVXLTVXCZYWTL7JA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142910119 (116h59m29.57399024s) [noise=true , address=104.36.82.37:4678 , piece_id=S5AV5WE4DOUZIRLT3GMKQLPLHARLF24HTVG3QNJ2GWZPYIC55PGQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142917733 (116h59m29.573997864s) [noise=true , address=66.23.193.175:4564 , piece_id=S4CHPP736M3D5GOURHJRONGXSYNTREBY3S243VJ6E5ZM75TDTLPA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143203881 (116h59m29.574284012s) [noise=true , address=64.49.13.140:4511 , piece_id=VTJ5NDREGII5G4IOP44JNYRJ5IFHHRP7S7OWTRFOGL4FCKBAUDYA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143239018 (116h59m29.574319149s) [noise=true , address=104.36.82.34:4560 , piece_id=4ZZFZKJRUXPMKOE2RDH6ZWFP3TVUOPSRQS7FAB2YANQ3MKA7JLWQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143481342 (116h59m29.574561473s) [noise=true , address=104.36.82.39:4727 , piece_id=UYFASTBHHBMDAY7ZUYHNSNSDDZMPPZ4AIR6X6HC6ERBXLXF4AOUQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143486071 (116h59m29.574566192s) [noise=true , address=104.36.82.23:4474 , piece_id=MDCBWJSKMP57ZE2V5WIKCG7QWIMRAOOIIFAAJQ5YXVWBEG2EG45Q, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143749975 (116h59m29.574830106s) [noise=true , address=77.90.31.11:4580 , piece_id=CBVCJWDPVPPQW2JIAV25UUFT7EVH62WJOP3FK4M7GUVJAO6P2QGA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.144014762 (116h59m29.575094893s) [noise=true , address=104.36.82.62:4412 , piece_id=CTYSBOF4JVZB5JTYNQDMP6Q5XK5KVETQH5SHHSR65K6HF3TZM2WQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.146502863 (116h59m29.577582994s) [noise=true , address=79.127.201.215:4358 , piece_id=5LNV5OH2YBCRCO7GJOK6Q3VSYC47ZJ4VTZQO3XVN7BLIRFRB4J5A, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.146602894 (116h59m29.577683025s) [noise=true , address=104.36.82.36:4642 , piece_id=TILD5YD3ZU4AOT5227X6PIEIKYXF52D2AZSNDWLOENLTQ4ZMVSLA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:36.312798371 (116h59m30.743878492s) [upload=121710]: Next returned nil segment Aug 13 14:25:36.312814072 (116h59m30.743894203s) [upload=121710]: Waiting for error group managing segments... Aug 13 14:25:36.608833520 (116h59m31.039913651s) [noise=true , address=66.23.193.175:4564 , piece_id=S4CHPP736M3D5GOURHJRONGXSYNTREBY3S243VJ6E5ZM75TDTLPA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.109920204 (116h59m31.541000325s) [noise=true , address=79.127.201.215:4358 , piece_id=5LNV5OH2YBCRCO7GJOK6Q3VSYC47ZJ4VTZQO3XVN7BLIRFRB4J5A, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.110478522 (116h59m31.541558653s) [noise=true , address=104.36.82.36:4642 , piece_id=TILD5YD3ZU4AOT5227X6PIEIKYXF52D2AZSNDWLOENLTQ4ZMVSLA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.127900888 (116h59m31.558981009s) [noise=true , address=77.90.31.11:4580 , piece_id=CBVCJWDPVPPQW2JIAV25UUFT7EVH62WJOP3FK4M7GUVJAO6P2QGA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.166070970 (116h59m31.597151101s) [noise=true , address=104.36.82.37:4678 , piece_id=S5AV5WE4DOUZIRLT3GMKQLPLHARLF24HTVG3QNJ2GWZPYIC55PGQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.167794408 (116h59m31.598874539s) [noise=true , address=104.36.82.19:4408 , piece_id=LT43HOEP3H4NTUKHSRUXPSRO4XBEJ6NOHIAPCBPZKVXPIW7SLLQQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.167973721 (116h59m31.599053852s) [noise=true , address=104.36.82.59:4328 , piece_id=EVWPCNQWESLH26NS62FKFXKOIW64GUDZFCABBK6H4FSYP35OA5OA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.195711503 (116h59m31.626791634s) [noise=true , address=104.36.82.34:4560 , piece_id=4ZZFZKJRUXPMKOE2RDH6ZWFP3TVUOPSRQS7FAB2YANQ3MKA7JLWQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.209376008 (116h59m31.640456129s) [noise=true , address=104.36.82.39:4727 , piece_id=UYFASTBHHBMDAY7ZUYHNSNSDDZMPPZ4AIR6X6HC6ERBXLXF4AOUQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.252511451 (116h59m31.683591582s) [noise=true , address=77.90.31.19:4846 , piece_id=3UYXBMMHS5MPUNXI6PY65K3AGHIIIVSGXJDEUVXLTVXCZYWTL7JA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.256659508 (116h59m31.687739639s) [noise=true , address=104.36.82.38:4719 , piece_id=PYZMWL3OZIULLOGDVKRF2RSLGLDGGWTCFWHQ66CPKPPEP6JGCVXQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.294487646 (116h59m31.725567777s) [noise=true , address=77.90.31.15:4702 , piece_id=SOXOEHUIJGYJKQ64YWBZMGAZMQJHR5RQMP3VFS5SGDSFM6J52ZMQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.310182075 (116h59m31.741262206s) [noise=true , address=104.36.82.62:4412 , piece_id=CTYSBOF4JVZB5JTYNQDMP6Q5XK5KVETQH5SHHSR65K6HF3TZM2WQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.514693066 (116h59m31.945773197s) [noise=true , address=104.36.82.23:4474 , piece_id=MDCBWJSKMP57ZE2V5WIKCG7QWIMRAOOIIFAAJQ5YXVWBEG2EG45Q, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.514716501 (116h59m31.945796632s) [seg_pos={0 0}, upload=121710]: Segment reached optimal threshold of 14 pieces. Aug 13 14:25:37.514831381 (116h59m31.945911512s) [noise=true , address=64.49.13.140:4511 , piece_id=VTJ5NDREGII5G4IOP44JNYRJ5IFHHRP7S7OWTRFOGL4FCKBAUDYA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: expected piece hash; context canceled; EOF Aug 13 14:25:37.514874733 (116h59m31.945954864s) [noise=true , address=104.36.82.56:4235 , piece_id=HS4RRPCLDB5ZMD2EJZZCTSCWVY3TWZC6QDVS4TRXRHO3EGZGR7UQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1a1sW1G1WPL5WQ2XxprWKQMsJkcb5ETSkk3ULp1TjRaZWptaT): protocol: expected piece hash; context canceled; EOF Aug 13 14:25:37.515019360 (116h59m31.946099492s) [noise=true , address=104.36.82.44:4874 , piece_id=C2ATKCPCO7P4KSHSMRRNGYCMH4IRDKUHL5ERDJKIJ4OECUCLQCFQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection; write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection Aug 13 14:25:37.515053225 (116h59m31.946133356s) [noise=true , address=104.36.82.60:4371 , piece_id=RTCVTBKO26FY7ZUNFYYJ2IOQ5XJCGG774ZP4WFLOVK5D47ULGXOA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KJP5DHmfnSFQF3Q74GzMUvcAz9zh16td4pnsqHAwToiM4D1vd): protocol: write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection; write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection Aug 13 14:25:37.515074867 (116h59m31.946154998s) [seg_pos={0 0}, upload=121710]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:25:37.515134190 (116h59m31.946214321s) [upload=121710]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:25:37.515138007 (116h59m31.946218138s) [upload=121710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:38.573880469 (116h59m33.0049606s) [upload=121710]: Done uploading segments... Aug 13 14:25:38.573909565 (116h59m33.004989686s) [upload=121710]: Upload finished. err: <nil> Duration: 6.290662441s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.................................X..X.............X.....X.X.] duration: 1.930429886s FAIL (2025-08-13 14:26:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 14:25:57.508778246 (116h59m51.939858377s) [upload=121717]: Starting upload Aug 13 14:25:57.508850885 (116h59m51.939931016s) [upload=121717]: Done starting upload Aug 13 14:25:57.508940917 (116h59m51.940021058s) [upload=121717]: Uploading segments... Aug 13 14:25:57.509065897 (116h59m51.940146038s) [upload=121717]: Got next segment. Inline: true Aug 13 14:25:57.509071818 (116h59m51.940151959s) [upload=121717]: Waiting for error group managing segments... Aug 13 14:25:57.509160628 (116h59m51.940240769s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:57.509169355 (116h59m51.940249496s) [upload=121717]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:25:57.509174675 (116h59m51.940254806s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:58.324969756 (116h59m52.756049887s) [upload=121717]: Done uploading segments... Aug 13 14:25:58.324989163 (116h59m52.756069294s) [upload=121717]: Upload finished. err: <nil> Duration:* 1.930454663s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................X.......X.X...............] duration: 6.252482543s FAIL (2025-08-13 14:26:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.252506438s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..................................X..X.............X.....X.X] duration: 1.962297449s FAIL (2025-08-13 14:25:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 14:25:57.508778246 (116h59m51.939858377s) [upload=121717]: Starting upload Aug 13 14:25:57.508850885 (116h59m51.939931016s) [upload=121717]: Done starting upload Aug 13 14:25:57.508940917 (116h59m51.940021058s) [upload=121717]: Uploading segments... Aug 13 14:25:57.509065897 (116h59m51.940146038s) [upload=121717]: Got next segment. Inline: true Aug 13 14:25:57.509071818 (116h59m51.940151959s) [upload=121717]: Waiting for error group managing segments... Aug 13 14:25:57.509160628 (116h59m51.940240769s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:57.509169355 (116h59m51.940249496s) [upload=121717]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:25:57.509174675 (116h59m51.940254806s) [upload=121717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:58.324969756 (116h59m52.756049887s) [upload=121717]: Done uploading segments... Aug 13 14:25:58.324989163 (116h59m52.756069294s) [upload=121717]: Upload finished. err: <nil> Duration:* 1.962315573s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X..................X.....X......X.....................X] duration: 5.297249084s FAIL (2025-08-13 14:25:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 14:25:34.670450049 (116h59m29.10153018s) [upload=121710]: Starting upload Aug 13 14:25:34.670500074 (116h59m29.101580205s) [upload=121710]: Done starting upload Aug 13 14:25:34.670554288 (116h59m29.101634409s) [upload=121710]: Uploading segments... Aug 13 14:25:34.670715417 (116h59m29.101795548s) [upload=121710]: Got next segment. Inline: false Aug 13 14:25:34.670762726 (116h59m29.101842857s) [upload=121710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:25:34.670769440 (116h59m29.101849631s) [upload=121710]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:25:35.141817037 (116h59m29.572897168s) [seg_pos={0 0}, upload=121710]: Begin upload segment... Aug 13 14:25:35.141863436 (116h59m29.572943567s) [seg_pos={0 0}, upload=121710]: Done begin upload segment. Aug 13 14:25:35.141887462 (116h59m29.572967593s) [noise=true , address=104.36.82.38:4719 , piece_id=PYZMWL3OZIULLOGDVKRF2RSLGLDGGWTCFWHQ66CPKPPEP6JGCVXQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.141902260 (116h59m29.572982391s) [noise=true , address=104.36.82.59:4328 , piece_id=EVWPCNQWESLH26NS62FKFXKOIW64GUDZFCABBK6H4FSYP35OA5OA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.141945994 (116h59m29.573026125s) [noise=true , address=104.36.82.56:4235 , piece_id=HS4RRPCLDB5ZMD2EJZZCTSCWVY3TWZC6QDVS4TRXRHO3EGZGR7UQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142200952 (116h59m29.573281083s) [noise=true , address=104.36.82.60:4371 , piece_id=RTCVTBKO26FY7ZUNFYYJ2IOQ5XJCGG774ZP4WFLOVK5D47ULGXOA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142305862 (116h59m29.573385993s) [noise=true , address=104.36.82.44:4874 , piece_id=C2ATKCPCO7P4KSHSMRRNGYCMH4IRDKUHL5ERDJKIJ4OECUCLQCFQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142439999 (116h59m29.57352013s) [noise=true , address=104.36.82.19:4408 , piece_id=LT43HOEP3H4NTUKHSRUXPSRO4XBEJ6NOHIAPCBPZKVXPIW7SLLQQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142605826 (116h59m29.573685957s) [noise=true , address=77.90.31.15:4702 , piece_id=SOXOEHUIJGYJKQ64YWBZMGAZMQJHR5RQMP3VFS5SGDSFM6J52ZMQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142682213 (116h59m29.573762344s) [noise=true , address=77.90.31.19:4846 , piece_id=3UYXBMMHS5MPUNXI6PY65K3AGHIIIVSGXJDEUVXLTVXCZYWTL7JA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142910119 (116h59m29.57399024s) [noise=true , address=104.36.82.37:4678 , piece_id=S5AV5WE4DOUZIRLT3GMKQLPLHARLF24HTVG3QNJ2GWZPYIC55PGQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.142917733 (116h59m29.573997864s) [noise=true , address=66.23.193.175:4564 , piece_id=S4CHPP736M3D5GOURHJRONGXSYNTREBY3S243VJ6E5ZM75TDTLPA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143203881 (116h59m29.574284012s) [noise=true , address=64.49.13.140:4511 , piece_id=VTJ5NDREGII5G4IOP44JNYRJ5IFHHRP7S7OWTRFOGL4FCKBAUDYA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143239018 (116h59m29.574319149s) [noise=true , address=104.36.82.34:4560 , piece_id=4ZZFZKJRUXPMKOE2RDH6ZWFP3TVUOPSRQS7FAB2YANQ3MKA7JLWQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143481342 (116h59m29.574561473s) [noise=true , address=104.36.82.39:4727 , piece_id=UYFASTBHHBMDAY7ZUYHNSNSDDZMPPZ4AIR6X6HC6ERBXLXF4AOUQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143486071 (116h59m29.574566192s) [noise=true , address=104.36.82.23:4474 , piece_id=MDCBWJSKMP57ZE2V5WIKCG7QWIMRAOOIIFAAJQ5YXVWBEG2EG45Q, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.143749975 (116h59m29.574830106s) [noise=true , address=77.90.31.11:4580 , piece_id=CBVCJWDPVPPQW2JIAV25UUFT7EVH62WJOP3FK4M7GUVJAO6P2QGA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.144014762 (116h59m29.575094893s) [noise=true , address=104.36.82.62:4412 , piece_id=CTYSBOF4JVZB5JTYNQDMP6Q5XK5KVETQH5SHHSR65K6HF3TZM2WQ, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.146502863 (116h59m29.577582994s) [noise=true , address=79.127.201.215:4358 , piece_id=5LNV5OH2YBCRCO7GJOK6Q3VSYC47ZJ4VTZQO3XVN7BLIRFRB4J5A, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:35.146602894 (116h59m29.577683025s) [noise=true , address=104.36.82.36:4642 , piece_id=TILD5YD3ZU4AOT5227X6PIEIKYXF52D2AZSNDWLOENLTQ4ZMVSLA, seg_pos={0 0}, upload=121710]: Uploading piece... Aug 13 14:25:36.312798371 (116h59m30.743878492s) [upload=121710]: Next returned nil segment Aug 13 14:25:36.312814072 (116h59m30.743894203s) [upload=121710]: Waiting for error group managing segments... Aug 13 14:25:36.608833520 (116h59m31.039913651s) [noise=true , address=66.23.193.175:4564 , piece_id=S4CHPP736M3D5GOURHJRONGXSYNTREBY3S243VJ6E5ZM75TDTLPA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.109920204 (116h59m31.541000325s) [noise=true , address=79.127.201.215:4358 , piece_id=5LNV5OH2YBCRCO7GJOK6Q3VSYC47ZJ4VTZQO3XVN7BLIRFRB4J5A, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.110478522 (116h59m31.541558653s) [noise=true , address=104.36.82.36:4642 , piece_id=TILD5YD3ZU4AOT5227X6PIEIKYXF52D2AZSNDWLOENLTQ4ZMVSLA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.127900888 (116h59m31.558981009s) [noise=true , address=77.90.31.11:4580 , piece_id=CBVCJWDPVPPQW2JIAV25UUFT7EVH62WJOP3FK4M7GUVJAO6P2QGA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.166070970 (116h59m31.597151101s) [noise=true , address=104.36.82.37:4678 , piece_id=S5AV5WE4DOUZIRLT3GMKQLPLHARLF24HTVG3QNJ2GWZPYIC55PGQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.167794408 (116h59m31.598874539s) [noise=true , address=104.36.82.19:4408 , piece_id=LT43HOEP3H4NTUKHSRUXPSRO4XBEJ6NOHIAPCBPZKVXPIW7SLLQQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.167973721 (116h59m31.599053852s) [noise=true , address=104.36.82.59:4328 , piece_id=EVWPCNQWESLH26NS62FKFXKOIW64GUDZFCABBK6H4FSYP35OA5OA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.195711503 (116h59m31.626791634s) [noise=true , address=104.36.82.34:4560 , piece_id=4ZZFZKJRUXPMKOE2RDH6ZWFP3TVUOPSRQS7FAB2YANQ3MKA7JLWQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.209376008 (116h59m31.640456129s) [noise=true , address=104.36.82.39:4727 , piece_id=UYFASTBHHBMDAY7ZUYHNSNSDDZMPPZ4AIR6X6HC6ERBXLXF4AOUQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.252511451 (116h59m31.683591582s) [noise=true , address=77.90.31.19:4846 , piece_id=3UYXBMMHS5MPUNXI6PY65K3AGHIIIVSGXJDEUVXLTVXCZYWTL7JA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.256659508 (116h59m31.687739639s) [noise=true , address=104.36.82.38:4719 , piece_id=PYZMWL3OZIULLOGDVKRF2RSLGLDGGWTCFWHQ66CPKPPEP6JGCVXQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.294487646 (116h59m31.725567777s) [noise=true , address=77.90.31.15:4702 , piece_id=SOXOEHUIJGYJKQ64YWBZMGAZMQJHR5RQMP3VFS5SGDSFM6J52ZMQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.310182075 (116h59m31.741262206s) [noise=true , address=104.36.82.62:4412 , piece_id=CTYSBOF4JVZB5JTYNQDMP6Q5XK5KVETQH5SHHSR65K6HF3TZM2WQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.514693066 (116h59m31.945773197s) [noise=true , address=104.36.82.23:4474 , piece_id=MDCBWJSKMP57ZE2V5WIKCG7QWIMRAOOIIFAAJQ5YXVWBEG2EG45Q, seg_pos={0 0}, upload=121710]: Done uploading piece. err: <nil> Aug 13 14:25:37.514716501 (116h59m31.945796632s) [seg_pos={0 0}, upload=121710]: Segment reached optimal threshold of 14 pieces. Aug 13 14:25:37.514831381 (116h59m31.945911512s) [noise=true , address=64.49.13.140:4511 , piece_id=VTJ5NDREGII5G4IOP44JNYRJ5IFHHRP7S7OWTRFOGL4FCKBAUDYA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: expected piece hash; context canceled; EOF Aug 13 14:25:37.514874733 (116h59m31.945954864s) [noise=true , address=104.36.82.56:4235 , piece_id=HS4RRPCLDB5ZMD2EJZZCTSCWVY3TWZC6QDVS4TRXRHO3EGZGR7UQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1a1sW1G1WPL5WQ2XxprWKQMsJkcb5ETSkk3ULp1TjRaZWptaT): protocol: expected piece hash; context canceled; EOF Aug 13 14:25:37.515019360 (116h59m31.946099492s) [noise=true , address=104.36.82.44:4874 , piece_id=C2ATKCPCO7P4KSHSMRRNGYCMH4IRDKUHL5ERDJKIJ4OECUCLQCFQ, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection; write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50256->104.36.82.44:4874: use of closed network connection Aug 13 14:25:37.515053225 (116h59m31.946133356s) [noise=true , address=104.36.82.60:4371 , piece_id=RTCVTBKO26FY7ZUNFYYJ2IOQ5XJCGG774ZP4WFLOVK5D47ULGXOA, seg_pos={0 0}, upload=121710]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KJP5DHmfnSFQF3Q74GzMUvcAz9zh16td4pnsqHAwToiM4D1vd): protocol: write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection; write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54100->104.36.82.60:4371: use of closed network connection Aug 13 14:25:37.515074867 (116h59m31.946154998s) [seg_pos={0 0}, upload=121710]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:25:37.515134190 (116h59m31.946214321s) [upload=121710]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:25:37.515138007 (116h59m31.946218138s) [upload=121710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:25:38.573880469 (116h59m33.0049606s) [upload=121710]: Done uploading segments... Aug 13 14:25:38.573909565 (116h59m33.004989686s) [upload=121710]: Upload finished. err: <nil> Duration: 5.297271957s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................X.......X.X..............] duration: 11.893629359s FAIL (2025-08-13 14:25:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.893653304s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X..................X.....X......X.....................] duration: 6.362705164s FAIL (2025-08-13 14:25:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.362731003s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...................................X..X.............X.....X.] duration: 1.926710641s FAIL (2025-08-13 14:24:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 14:23:46.826324057 (116h57m41.257404188s) [upload=121683]: Starting upload Aug 13 14:23:46.826373591 (116h57m41.257453772s) [upload=121683]: Done starting upload Aug 13 14:23:46.826451750 (116h57m41.257531881s) [upload=121683]: Uploading segments... Aug 13 14:23:46.826484884 (116h57m41.257565015s) [upload=121683]: Got next segment. Inline: true Aug 13 14:23:46.826488812 (116h57m41.257568943s) [upload=121683]: Waiting for error group managing segments... Aug 13 14:23:46.826525432 (116h57m41.257605563s) [upload=121683]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:23:46.826529610 (116h57m41.257609741s) [upload=121683]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:23:46.826531964 (116h57m41.257612085s) [upload=121683]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:23:48.416127264 (116h57m42.847207395s) [upload=121683]: Done uploading segments... Aug 13 14:23:48.416147593 (116h57m42.847227724s) [upload=121683]: Upload finished. err: <nil> Duration:* 1.926735317s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................X.......X.X.............] duration: 6.953212974s FAIL (2025-08-13 14:24:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.953234605s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X.......X.X............] duration: 10.651236271s FAIL (2025-08-13 14:24:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.651270006s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X..................X.....X......X....................] duration: 6.47171118s FAIL (2025-08-13 14:24:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.47175845s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....................................X..X.............X.....X] duration: 2.751309797s FAIL (2025-08-13 14:23:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 14:23:46.826324057 (116h57m41.257404188s) [upload=121683]: Starting upload Aug 13 14:23:46.826373591 (116h57m41.257453772s) [upload=121683]: Done starting upload Aug 13 14:23:46.826451750 (116h57m41.257531881s) [upload=121683]: Uploading segments... Aug 13 14:23:46.826484884 (116h57m41.257565015s) [upload=121683]: Got next segment. Inline: true Aug 13 14:23:46.826488812 (116h57m41.257568943s) [upload=121683]: Waiting for error group managing segments... Aug 13 14:23:46.826525432 (116h57m41.257605563s) [upload=121683]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:23:46.826529610 (116h57m41.257609741s) [upload=121683]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:23:46.826531964 (116h57m41.257612085s) [upload=121683]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:23:48.416127264 (116h57m42.847207395s) [upload=121683]: Done uploading segments... Aug 13 14:23:48.416147593 (116h57m42.847227724s) [upload=121683]: Upload finished. err: <nil> Duration:* 2.751333893s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X..................X.....X......X...................] duration: 6.462951671s FAIL (2025-08-13 14:23:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.462975124s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................X.......X.X...........] duration: 6.32206719s FAIL (2025-08-13 14:23:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.322077259s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X..................X.....X......X..................] duration: 6.124287158s FAIL (2025-08-13 14:23:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.124311965s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X..X.............X.....] duration: 2.045900793s FAIL (2025-08-13 14:22:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3c3c822f03c4eb3eba81f7999f4e1761cf64f46faf175cce354f0f00df761f8e != before cc5575e043c348e74351d716fe0f992dd4fd69e1bab0e50ab4d249b5a511ba65 ======LOGS====== Aug 13 14:18:32.695761218 (116h52m27.126841359s) [upload=121592]: Starting upload Aug 13 14:18:32.695827415 (116h52m27.126907546s) [upload=121592]: Done starting upload Aug 13 14:18:32.695922797 (116h52m27.127002928s) [upload=121592]: Uploading segments... Aug 13 14:18:32.695979476 (116h52m27.127059607s) [upload=121592]: Got next segment. Inline: true Aug 13 14:18:32.695984897 (116h52m27.127065038s) [upload=121592]: Waiting for error group managing segments... Aug 13 14:18:32.696032437 (116h52m27.127112568s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:18:32.696037738 (116h52m27.127117869s) [upload=121592]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:18:32.696041164 (116h52m27.127121295s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:18:32.984899624 (116h52m27.415979755s) [upload=121592]: Done uploading segments... Aug 13 14:18:32.984916205 (116h52m27.415996336s) [upload=121592]: Upload finished. err: <nil> Duration:* 2.045927603s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X..................X.....X......X.................] duration: 6.262535177s FAIL (2025-08-13 14:22:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.262572258s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X.......X.X..........] duration: 12.549719264s FAIL (2025-08-13 14:22:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.549744162s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................X.......X.X.........] duration: 6.503768635s FAIL (2025-08-13 14:22:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.503804213s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................................X..X.............X....] duration: 2.025608904s FAIL (2025-08-13 14:21:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3c3c822f03c4eb3eba81f7999f4e1761cf64f46faf175cce354f0f00df761f8e != before cc5575e043c348e74351d716fe0f992dd4fd69e1bab0e50ab4d249b5a511ba65 ======LOGS====== Aug 13 14:18:32.695761218 (116h52m27.126841359s) [upload=121592]: Starting upload Aug 13 14:18:32.695827415 (116h52m27.126907546s) [upload=121592]: Done starting upload Aug 13 14:18:32.695922797 (116h52m27.127002928s) [upload=121592]: Uploading segments... Aug 13 14:18:32.695979476 (116h52m27.127059607s) [upload=121592]: Got next segment. Inline: true Aug 13 14:18:32.695984897 (116h52m27.127065038s) [upload=121592]: Waiting for error group managing segments... Aug 13 14:18:32.696032437 (116h52m27.127112568s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:18:32.696037738 (116h52m27.127117869s) [upload=121592]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:18:32.696041164 (116h52m27.127121295s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:18:32.984899624 (116h52m27.415979755s) [upload=121592]: Done uploading segments... Aug 13 14:18:32.984916205 (116h52m27.415996336s) [upload=121592]: Upload finished. err: <nil> Duration:* 2.025636487s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X..................X.....X......X................] duration: 7.083988604s FAIL (2025-08-13 14:21:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 7.084011728s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.......X.X........] duration: 7.408238515s FAIL (2025-08-13 14:21:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.408260528s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X..................X.....X......X...............] duration: 6.882444559s FAIL (2025-08-13 14:20:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.882468956s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X.......X.X.......] duration: 6.90501518s FAIL (2025-08-13 14:20:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.905041699s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X..X.............X...] duration: 1.938382734s FAIL (2025-08-13 14:20:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3c3c822f03c4eb3eba81f7999f4e1761cf64f46faf175cce354f0f00df761f8e != before cc5575e043c348e74351d716fe0f992dd4fd69e1bab0e50ab4d249b5a511ba65 ======LOGS====== Aug 13 14:18:32.695761218 (116h52m27.126841359s) [upload=121592]: Starting upload Aug 13 14:18:32.695827415 (116h52m27.126907546s) [upload=121592]: Done starting upload Aug 13 14:18:32.695922797 (116h52m27.127002928s) [upload=121592]: Uploading segments... Aug 13 14:18:32.695979476 (116h52m27.127059607s) [upload=121592]: Got next segment. Inline: true Aug 13 14:18:32.695984897 (116h52m27.127065038s) [upload=121592]: Waiting for error group managing segments... Aug 13 14:18:32.696032437 (116h52m27.127112568s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:18:32.696037738 (116h52m27.127117869s) [upload=121592]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:18:32.696041164 (116h52m27.127121295s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:18:32.984899624 (116h52m27.415979755s) [upload=121592]: Done uploading segments... Aug 13 14:18:32.984916205 (116h52m27.415996336s) [upload=121592]: Upload finished. err: <nil> Duration:* 1.938408063s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X..................X.....X......X..............] duration: 6.146593578s FAIL (2025-08-13 14:20:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.146620219s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X..................X.....X......X.............] duration: 6.378259286s FAIL (2025-08-13 14:19:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.378279484s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X.......X.X......] duration: 9.939363228s FAIL (2025-08-13 14:19:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.939383396s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................................X..X.............X..] duration: 1.945289405s FAIL (2025-08-13 14:19:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3c3c822f03c4eb3eba81f7999f4e1761cf64f46faf175cce354f0f00df761f8e != before cc5575e043c348e74351d716fe0f992dd4fd69e1bab0e50ab4d249b5a511ba65 ======LOGS====== Aug 13 14:18:32.695761218 (116h52m27.126841359s) [upload=121592]: Starting upload Aug 13 14:18:32.695827415 (116h52m27.126907546s) [upload=121592]: Done starting upload Aug 13 14:18:32.695922797 (116h52m27.127002928s) [upload=121592]: Uploading segments... Aug 13 14:18:32.695979476 (116h52m27.127059607s) [upload=121592]: Got next segment. Inline: true Aug 13 14:18:32.695984897 (116h52m27.127065038s) [upload=121592]: Waiting for error group managing segments... Aug 13 14:18:32.696032437 (116h52m27.127112568s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:18:32.696037738 (116h52m27.127117869s) [upload=121592]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:18:32.696041164 (116h52m27.127121295s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:18:32.984899624 (116h52m27.415979755s) [upload=121592]: Done uploading segments... Aug 13 14:18:32.984916205 (116h52m27.415996336s) [upload=121592]: Upload finished. err: <nil> Duration:* 1.945320325s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X..................X.....X......X............] duration: 6.92709446s FAIL (2025-08-13 14:19:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 6.927123675s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X.......X.X.....] duration: 6.522953867s FAIL (2025-08-13 14:19:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.522977963s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X..X.............X.] duration: 1.958176946s FAIL (2025-08-13 14:19:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3c3c822f03c4eb3eba81f7999f4e1761cf64f46faf175cce354f0f00df761f8e != before cc5575e043c348e74351d716fe0f992dd4fd69e1bab0e50ab4d249b5a511ba65 ======LOGS====== Aug 13 14:18:32.695761218 (116h52m27.126841359s) [upload=121592]: Starting upload Aug 13 14:18:32.695827415 (116h52m27.126907546s) [upload=121592]: Done starting upload Aug 13 14:18:32.695922797 (116h52m27.127002928s) [upload=121592]: Uploading segments... Aug 13 14:18:32.695979476 (116h52m27.127059607s) [upload=121592]: Got next segment. Inline: true Aug 13 14:18:32.695984897 (116h52m27.127065038s) [upload=121592]: Waiting for error group managing segments... Aug 13 14:18:32.696032437 (116h52m27.127112568s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:18:32.696037738 (116h52m27.127117869s) [upload=121592]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:18:32.696041164 (116h52m27.127121295s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:18:32.984899624 (116h52m27.415979755s) [upload=121592]: Done uploading segments... Aug 13 14:18:32.984916205 (116h52m27.415996336s) [upload=121592]: Upload finished. err: <nil> Duration:* 1.958204991s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X..................X.....X......X...........] duration: 5.747147461s FAIL (2025-08-13 14:18:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 5.747173911s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X..X.............X] duration: 1.762394061s FAIL (2025-08-13 14:18:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 3c3c822f03c4eb3eba81f7999f4e1761cf64f46faf175cce354f0f00df761f8e != before cc5575e043c348e74351d716fe0f992dd4fd69e1bab0e50ab4d249b5a511ba65 ======LOGS====== Aug 13 14:18:32.695761218 (116h52m27.126841359s) [upload=121592]: Starting upload Aug 13 14:18:32.695827415 (116h52m27.126907546s) [upload=121592]: Done starting upload Aug 13 14:18:32.695922797 (116h52m27.127002928s) [upload=121592]: Uploading segments... Aug 13 14:18:32.695979476 (116h52m27.127059607s) [upload=121592]: Got next segment. Inline: true Aug 13 14:18:32.695984897 (116h52m27.127065038s) [upload=121592]: Waiting for error group managing segments... Aug 13 14:18:32.696032437 (116h52m27.127112568s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:18:32.696037738 (116h52m27.127117869s) [upload=121592]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:18:32.696041164 (116h52m27.127121295s) [upload=121592]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:18:32.984899624 (116h52m27.415979755s) [upload=121592]: Done uploading segments... Aug 13 14:18:32.984916205 (116h52m27.415996336s) [upload=121592]: Upload finished. err: <nil> Duration:* 1.762418447s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X.......X.X....] duration: 12.019377258s FAIL (2025-08-13 14:18:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.019413838s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................X.......X.X...] duration: 11.388699909s FAIL (2025-08-13 14:18:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.388713866s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X................X..................X.....X......X..........] duration: 7.274659068s FAIL (2025-08-13 14:17:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 7.274683625s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X..X.............] duration: 1.954338862s FAIL (2025-08-13 14:17:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 14:07:38.284942970 (116h41m32.716023101s) [upload=121412]: Starting upload Aug 13 14:07:38.284983688 (116h41m32.716063819s) [upload=121412]: Done starting upload Aug 13 14:07:38.285029646 (116h41m32.716109767s) [upload=121412]: Uploading segments... Aug 13 14:07:38.285084812 (116h41m32.716164943s) [upload=121412]: Got next segment. Inline: true Aug 13 14:07:38.285090262 (116h41m32.716170383s) [upload=121412]: Waiting for error group managing segments... Aug 13 14:07:38.285126462 (116h41m32.716206593s) [upload=121412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:07:38.285130710 (116h41m32.716210841s) [upload=121412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:07:38.285133375 (116h41m32.716213506s) [upload=121412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:07:38.599914816 (116h41m33.030994947s) [upload=121412]: Done uploading segments... Aug 13 14:07:38.599929904 (116h41m33.031010035s) [upload=121412]: Upload finished. err: <nil> Duration:* 1.954361735s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X.......X.X..] duration: 6.573332248s FAIL (2025-08-13 14:17:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.573354711s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X..X............] duration: 1.923435494s FAIL (2025-08-13 14:17:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 14:07:38.284942970 (116h41m32.716023101s) [upload=121412]: Starting upload Aug 13 14:07:38.284983688 (116h41m32.716063819s) [upload=121412]: Done starting upload Aug 13 14:07:38.285029646 (116h41m32.716109767s) [upload=121412]: Uploading segments... Aug 13 14:07:38.285084812 (116h41m32.716164943s) [upload=121412]: Got next segment. Inline: true Aug 13 14:07:38.285090262 (116h41m32.716170383s) [upload=121412]: Waiting for error group managing segments... Aug 13 14:07:38.285126462 (116h41m32.716206593s) [upload=121412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:07:38.285130710 (116h41m32.716210841s) [upload=121412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:07:38.285133375 (116h41m32.716213506s) [upload=121412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:07:38.599914816 (116h41m33.030994947s) [upload=121412]: Done uploading segments... Aug 13 14:07:38.599929904 (116h41m33.031010035s) [upload=121412]: Upload finished. err: <nil> Duration:* 1.923462354s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................X.......X.X.] duration: 18.365094702s FAIL (2025-08-13 14:16:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.365117195s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X................X..................X.....X......X.........] duration: 7.08261972s FAIL (2025-08-13 14:16:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 7.082641952s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X..X...........] duration: 1.941473466s FAIL (2025-08-13 14:16:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 14:07:38.284942970 (116h41m32.716023101s) [upload=121412]: Starting upload Aug 13 14:07:38.284983688 (116h41m32.716063819s) [upload=121412]: Done starting upload Aug 13 14:07:38.285029646 (116h41m32.716109767s) [upload=121412]: Uploading segments... Aug 13 14:07:38.285084812 (116h41m32.716164943s) [upload=121412]: Got next segment. Inline: true Aug 13 14:07:38.285090262 (116h41m32.716170383s) [upload=121412]: Waiting for error group managing segments... Aug 13 14:07:38.285126462 (116h41m32.716206593s) [upload=121412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:07:38.285130710 (116h41m32.716210841s) [upload=121412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 14:07:38.285133375 (116h41m32.716213506s) [upload=121412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:07:38.599914816 (116h41m33.030994947s) [upload=121412]: Done uploading segments... Aug 13 14:07:38.599929904 (116h41m33.031010035s) [upload=121412]: Upload finished. err: <nil> Duration:* 1.941509365s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................X.......X.X] duration: 30.001607349s FAIL (2025-08-13 14:16:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001633079s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X................X..................X.....X......X........] duration: 7.866327422s FAIL (2025-08-13 14:15:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 14:09:34.818248297 (116h43m29.249328438s) [upload=121446]: Starting upload Aug 13 14:09:34.818322739 (116h43m29.24940287s) [upload=121446]: Done starting upload Aug 13 14:09:34.818406669 (116h43m29.2494868s) [upload=121446]: Uploading segments... Aug 13 14:09:34.818602484 (116h43m29.249682615s) [upload=121446]: Got next segment. Inline: false Aug 13 14:09:34.818630367 (116h43m29.249710498s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 14:09:34.818636149 (116h43m29.24971628s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 14:09:35.289892981 (116h43m29.720973112s) [seg_pos={0 0}, upload=121446]: Begin upload segment... Aug 13 14:09:35.289944780 (116h43m29.721024911s) [seg_pos={0 0}, upload=121446]: Done begin upload segment. Aug 13 14:09:35.289965510 (116h43m29.721045651s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290018711 (116h43m29.721098853s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290342441 (116h43m29.721422572s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290421281 (116h43m29.721501412s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290635752 (116h43m29.721715883s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290689905 (116h43m29.721770036s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290960433 (116h43m29.722040574s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.290970191 (116h43m29.722050322s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291233516 (116h43m29.722313647s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291464668 (116h43m29.7225448s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.291707673 (116h43m29.722787804s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292060508 (116h43m29.723140659s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292519296 (116h43m29.723599427s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.292907729 (116h43m29.72398786s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293337231 (116h43m29.724417362s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.293739571 (116h43m29.724819702s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294089069 (116h43m29.7251692s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:35.294322787 (116h43m29.725402918s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Uploading piece... Aug 13 14:09:36.416845628 (116h43m30.847925759s) [upload=121446]: Next returned nil segment Aug 13 14:09:36.416865557 (116h43m30.847945688s) [upload=121446]: Waiting for error group managing segments... Aug 13 14:09:36.717396961 (116h43m31.148477092s) [noise=true , address=66.23.193.175:4556 , piece_id=KGDEFMTY2LG53KBZJRZSO2ARDCWMIMJKQWD3K4SKV4VZQYNB6FOA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:36.791714307 (116h43m31.222794448s) [noise=true , address=64.49.13.140:4516 , piece_id=3DTSLP3ORUADXQZKMDOUKQV44CEAQOJAARKHVV77N7JX5FYKZ5YQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.117053769 (116h43m31.5481339s) [noise=true , address=79.127.220.101:4059 , piece_id=SSN4B6RJIVZEAMOXATKY2RDC5W3A4ORNYCYFMNE46LBJ7OZ4WKEQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.255289944 (116h43m31.686370065s) [noise=true , address=77.90.31.18:4822 , piece_id=UVQHTMGHOKJTM5KZOB2O7GN7G6WJRAQRLAOYN5CT5BSL3ZV6VJ7Q, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.262040811 (116h43m31.693120942s) [noise=true , address=104.36.82.13:4192 , piece_id=PHCGBVB45BMXZ642SQF662QZME2CUV5PM7PNVVZ25YHNLLXFHS3A, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.268595513 (116h43m31.699675644s) [noise=true , address=104.36.82.60:4365 , piece_id=NEUQUDAMZQ4XBP6725F5KSB2S3WWXQXNFBCXXR3J4XMFQ6KMJ5CA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.284378842 (116h43m31.715458973s) [noise=true , address=104.36.82.23:4447 , piece_id=F2HEWKEPWWMKIQCZGXG5SNFKHLA7EOKDNF6CGRLRGHGGPVVUUUFQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.285953765 (116h43m31.717033896s) [noise=true , address=77.90.31.16:4752 , piece_id=PMWTFQWZKBWBDL2LOHSCHA2RXUB5URSWZMD4T3LOSMEYZCRVPLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.293920148 (116h43m31.725000269s) [noise=true , address=104.36.82.4:4823 , piece_id=5CCBHRTJNWIHXVBCTHAI4VHVK54L75XW6PWHVKGZHQMRLZEIECUQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.315915523 (116h43m31.746995654s) [noise=true , address=104.36.82.8:4976 , piece_id=I6MDRJUEBPHMJJ2F3AFOOZAZNINOAUSQ3OQRA7MAB63XPKV3W2CQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.319160061 (116h43m31.750240202s) [noise=true , address=104.36.82.39:4757 , piece_id=364JYLBIVJCTNXYPSN55U67OHAMPPEGYMHHZPSE24HK4CUHBDLUA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337210758 (116h43m31.768290889s) [noise=true , address=79.127.201.215:4343 , piece_id=RZHQV2GTDJU5BLQNTQ73O67FZULFEW6XKZYHDNWBS2JJUOWTXKEA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.337841605 (116h43m31.768921736s) [noise=true , address=104.36.82.38:4703 , piece_id=CFLM2ALUZVDZB2IAWIAGCHYSNI3ESTNE5UCBI73JVP6XX7S3RLYA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346344885 (116h43m31.777425016s) [noise=true , address=104.36.82.37:4682 , piece_id=HH34KLCRIYO47NWSNKY3EKNP3SMHGP6T6HKMRCXWF6PLV76SG4KQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: <nil> Aug 13 14:09:37.346389560 (116h43m31.777469691s) [seg_pos={0 0}, upload=121446]: Segment reached optimal threshold of 14 pieces. Aug 13 14:09:37.346525941 (116h43m31.777606072s) [noise=true , address=104.36.82.61:4375 , piece_id=6JYIDREVXCUSD2WVKPU3F57MCOLAVDK5OEZW6YAXQXJIHBMJ6FMQ, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346582569 (116h43m31.7776627s) [noise=true , address=104.36.82.59:4316 , piece_id=TTNLZQJG6N52EFDJBW4SPAYB6CWTI5OMCIJVLGQDOA6H7QAGGZAA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124WiP4Umxs7oHKtCpaQrysLxRDsrmqkyTCs2Ps38ZjM2LPvVhn): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346635120 (116h43m31.777715251s) [noise=true , address=104.36.82.44:4886 , piece_id=TBRJJVGNHZGDAZHW4M74DPSQ4NBD5LATLNEF2JHICAHWQSCPLMPA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346764578 (116h43m31.777844709s) [noise=true , address=104.36.82.36:4625 , piece_id=S5MT6QXSOCEXXYJNMFKUCLQABW3BYSL54TK25L33DHWXZ6OIGRIA, seg_pos={0 0}, upload=121446]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF): protocol: expected piece hash; context canceled; EOF Aug 13 14:09:37.346825424 (116h43m31.777905545s) [seg_pos={0 0}, upload=121446]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 14:09:37.346882784 (116h43m31.777962915s) [upload=121446]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 14:09:37.346886651 (116h43m31.777966782s) [upload=121446]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 14:09:37.656424119 (116h43m32.08750425s) [upload=121446]: Done uploading segments... Aug 13 14:09:37.656441832 (116h43m32.087521963s) [upload=121446]: Upload finished. err: <nil> Duration: 7.866361577s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)