Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.X.................X.............X...........X........X...] duration: 5.959271899s FAIL (2025-08-22 18:55:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0ef7802de24bb1cd2ebf0338e0eedc506fc1c735ce91020b8e151bba16536846 != before 09db45af19e4629af7d1bfdd0f2973cd9e14bf5deecf68914a566b0a052bf7b9 ======LOGS====== Aug 22 18:53:18.611368343 (337h27m13.042448474s) [upload=351066]: Starting upload Aug 22 18:53:18.611414451 (337h27m13.042494582s) [upload=351066]: Done starting upload Aug 22 18:53:18.611493242 (337h27m13.042573383s) [upload=351066]: Uploading segments... Aug 22 18:53:18.611592071 (337h27m13.042672202s) [upload=351066]: Got next segment. Inline: false Aug 22 18:53:18.611620335 (337h27m13.042700466s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:53:18.611624873 (337h27m13.042705004s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:53:19.067533922 (337h27m13.498614053s) [seg_pos={0 0}, upload=351066]: Begin upload segment... Aug 22 18:53:19.067578687 (337h27m13.498658818s) [seg_pos={0 0}, upload=351066]: Done begin upload segment. Aug 22 18:53:19.067598065 (337h27m13.498678196s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.067955498 (337h27m13.499035629s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068223632 (337h27m13.499303763s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068535899 (337h27m13.49961603s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068796498 (337h27m13.499876619s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.069036867 (337h27m13.500116998s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070079632 (337h27m13.501159763s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070493725 (337h27m13.501573847s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070784561 (337h27m13.501864692s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071097190 (337h27m13.502177321s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071381353 (337h27m13.502461485s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071623136 (337h27m13.502703267s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071868736 (337h27m13.502948867s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072111000 (337h27m13.503191131s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072363032 (337h27m13.503443163s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072631386 (337h27m13.503711517s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072872667 (337h27m13.503952798s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.073093810 (337h27m13.504173931s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:20.595594732 (337h27m15.026674853s) [upload=351066]: Next returned nil segment Aug 22 18:53:20.595612135 (337h27m15.026692266s) [upload=351066]: Waiting for error group managing segments... Aug 22 18:53:21.007836407 (337h27m15.438916578s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.025911572 (337h27m15.456991703s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.028467853 (337h27m15.459547984s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.047663913 (337h27m15.478744044s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.053450645 (337h27m15.484530786s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.069484074 (337h27m15.500564205s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.096650765 (337h27m15.527730896s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.105896205 (337h27m15.536976336s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.126992952 (337h27m15.558073083s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.181061637 (337h27m15.612141768s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.192320048 (337h27m15.623400179s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.196002525 (337h27m15.627082656s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.244578892 (337h27m15.675659023s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317255260 (337h27m15.748335391s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317286951 (337h27m15.748367072s) [seg_pos={0 0}, upload=351066]: Segment reached optimal threshold of 14 pieces. Aug 22 18:53:21.317387413 (337h27m15.748467544s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317447548 (337h27m15.748527679s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123aoo9g11aJP9hiL8shu2dx44tjyKzEWqPUjHUdXtR7LLFyng8): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317498345 (337h27m15.748578466s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317580633 (337h27m15.748660764s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DdwuGVzn339aKBRre1NvC9rUwSbL8v4GLwWj5TZBRP6RvLDnF): protocol: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection Aug 22 18:53:21.317604088 (337h27m15.748684219s) [seg_pos={0 0}, upload=351066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:53:21.317678330 (337h27m15.748758461s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:53:21.317682508 (337h27m15.748762649s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:53:21.615966306 (337h27m16.047046437s) [upload=351066]: Done uploading segments... Aug 22 18:53:21.615987125 (337h27m16.047067306s) [upload=351066]: Upload finished. err: <nil> Duration: 5.959292217s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.962373124s FAIL (2025-08-22 18:55:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.962401809s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X.X.................X.............X...........X........X..] duration: 5.94574594s FAIL (2025-08-22 18:55:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0ef7802de24bb1cd2ebf0338e0eedc506fc1c735ce91020b8e151bba16536846 != before 09db45af19e4629af7d1bfdd0f2973cd9e14bf5deecf68914a566b0a052bf7b9 ======LOGS====== Aug 22 18:53:18.611368343 (337h27m13.042448474s) [upload=351066]: Starting upload Aug 22 18:53:18.611414451 (337h27m13.042494582s) [upload=351066]: Done starting upload Aug 22 18:53:18.611493242 (337h27m13.042573383s) [upload=351066]: Uploading segments... Aug 22 18:53:18.611592071 (337h27m13.042672202s) [upload=351066]: Got next segment. Inline: false Aug 22 18:53:18.611620335 (337h27m13.042700466s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:53:18.611624873 (337h27m13.042705004s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:53:19.067533922 (337h27m13.498614053s) [seg_pos={0 0}, upload=351066]: Begin upload segment... Aug 22 18:53:19.067578687 (337h27m13.498658818s) [seg_pos={0 0}, upload=351066]: Done begin upload segment. Aug 22 18:53:19.067598065 (337h27m13.498678196s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.067955498 (337h27m13.499035629s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068223632 (337h27m13.499303763s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068535899 (337h27m13.49961603s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068796498 (337h27m13.499876619s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.069036867 (337h27m13.500116998s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070079632 (337h27m13.501159763s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070493725 (337h27m13.501573847s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070784561 (337h27m13.501864692s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071097190 (337h27m13.502177321s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071381353 (337h27m13.502461485s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071623136 (337h27m13.502703267s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071868736 (337h27m13.502948867s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072111000 (337h27m13.503191131s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072363032 (337h27m13.503443163s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072631386 (337h27m13.503711517s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072872667 (337h27m13.503952798s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.073093810 (337h27m13.504173931s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:20.595594732 (337h27m15.026674853s) [upload=351066]: Next returned nil segment Aug 22 18:53:20.595612135 (337h27m15.026692266s) [upload=351066]: Waiting for error group managing segments... Aug 22 18:53:21.007836407 (337h27m15.438916578s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.025911572 (337h27m15.456991703s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.028467853 (337h27m15.459547984s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.047663913 (337h27m15.478744044s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.053450645 (337h27m15.484530786s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.069484074 (337h27m15.500564205s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.096650765 (337h27m15.527730896s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.105896205 (337h27m15.536976336s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.126992952 (337h27m15.558073083s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.181061637 (337h27m15.612141768s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.192320048 (337h27m15.623400179s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.196002525 (337h27m15.627082656s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.244578892 (337h27m15.675659023s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317255260 (337h27m15.748335391s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317286951 (337h27m15.748367072s) [seg_pos={0 0}, upload=351066]: Segment reached optimal threshold of 14 pieces. Aug 22 18:53:21.317387413 (337h27m15.748467544s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317447548 (337h27m15.748527679s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123aoo9g11aJP9hiL8shu2dx44tjyKzEWqPUjHUdXtR7LLFyng8): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317498345 (337h27m15.748578466s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317580633 (337h27m15.748660764s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DdwuGVzn339aKBRre1NvC9rUwSbL8v4GLwWj5TZBRP6RvLDnF): protocol: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection Aug 22 18:53:21.317604088 (337h27m15.748684219s) [seg_pos={0 0}, upload=351066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:53:21.317678330 (337h27m15.748758461s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:53:21.317682508 (337h27m15.748762649s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:53:21.615966306 (337h27m16.047046437s) [upload=351066]: Done uploading segments... Aug 22 18:53:21.615987125 (337h27m16.047067306s) [upload=351066]: Upload finished. err: <nil> Duration: 5.945775627s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X.....................................X................] duration: 1.866272234s FAIL (2025-08-22 18:54:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 1.866302953s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.098763073s FAIL (2025-08-22 18:54:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.098785535s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X.X.................X.............X...........X........X.] duration: 5.992653102s FAIL (2025-08-22 18:54:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0ef7802de24bb1cd2ebf0338e0eedc506fc1c735ce91020b8e151bba16536846 != before 09db45af19e4629af7d1bfdd0f2973cd9e14bf5deecf68914a566b0a052bf7b9 ======LOGS====== Aug 22 18:53:18.611368343 (337h27m13.042448474s) [upload=351066]: Starting upload Aug 22 18:53:18.611414451 (337h27m13.042494582s) [upload=351066]: Done starting upload Aug 22 18:53:18.611493242 (337h27m13.042573383s) [upload=351066]: Uploading segments... Aug 22 18:53:18.611592071 (337h27m13.042672202s) [upload=351066]: Got next segment. Inline: false Aug 22 18:53:18.611620335 (337h27m13.042700466s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:53:18.611624873 (337h27m13.042705004s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:53:19.067533922 (337h27m13.498614053s) [seg_pos={0 0}, upload=351066]: Begin upload segment... Aug 22 18:53:19.067578687 (337h27m13.498658818s) [seg_pos={0 0}, upload=351066]: Done begin upload segment. Aug 22 18:53:19.067598065 (337h27m13.498678196s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.067955498 (337h27m13.499035629s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068223632 (337h27m13.499303763s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068535899 (337h27m13.49961603s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068796498 (337h27m13.499876619s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.069036867 (337h27m13.500116998s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070079632 (337h27m13.501159763s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070493725 (337h27m13.501573847s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070784561 (337h27m13.501864692s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071097190 (337h27m13.502177321s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071381353 (337h27m13.502461485s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071623136 (337h27m13.502703267s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071868736 (337h27m13.502948867s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072111000 (337h27m13.503191131s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072363032 (337h27m13.503443163s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072631386 (337h27m13.503711517s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072872667 (337h27m13.503952798s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.073093810 (337h27m13.504173931s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:20.595594732 (337h27m15.026674853s) [upload=351066]: Next returned nil segment Aug 22 18:53:20.595612135 (337h27m15.026692266s) [upload=351066]: Waiting for error group managing segments... Aug 22 18:53:21.007836407 (337h27m15.438916578s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.025911572 (337h27m15.456991703s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.028467853 (337h27m15.459547984s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.047663913 (337h27m15.478744044s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.053450645 (337h27m15.484530786s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.069484074 (337h27m15.500564205s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.096650765 (337h27m15.527730896s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.105896205 (337h27m15.536976336s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.126992952 (337h27m15.558073083s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.181061637 (337h27m15.612141768s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.192320048 (337h27m15.623400179s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.196002525 (337h27m15.627082656s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.244578892 (337h27m15.675659023s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317255260 (337h27m15.748335391s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317286951 (337h27m15.748367072s) [seg_pos={0 0}, upload=351066]: Segment reached optimal threshold of 14 pieces. Aug 22 18:53:21.317387413 (337h27m15.748467544s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317447548 (337h27m15.748527679s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123aoo9g11aJP9hiL8shu2dx44tjyKzEWqPUjHUdXtR7LLFyng8): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317498345 (337h27m15.748578466s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317580633 (337h27m15.748660764s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DdwuGVzn339aKBRre1NvC9rUwSbL8v4GLwWj5TZBRP6RvLDnF): protocol: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection Aug 22 18:53:21.317604088 (337h27m15.748684219s) [seg_pos={0 0}, upload=351066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:53:21.317678330 (337h27m15.748758461s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:53:21.317682508 (337h27m15.748762649s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:53:21.615966306 (337h27m16.047046437s) [upload=351066]: Done uploading segments... Aug 22 18:53:21.615987125 (337h27m16.047067306s) [upload=351066]: Upload finished. err: <nil> Duration: 5.992680705s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X.....................................X...............] duration: 2.011194339s FAIL (2025-08-22 18:54:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.011234395s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.99997868s FAIL (2025-08-22 18:53:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.000004661s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.144596439s FAIL (2025-08-22 18:53:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.144618492s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [X...X.X.................X.............X...........X........X] duration: 5.983735439s FAIL (2025-08-22 18:53:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 0ef7802de24bb1cd2ebf0338e0eedc506fc1c735ce91020b8e151bba16536846 != before 09db45af19e4629af7d1bfdd0f2973cd9e14bf5deecf68914a566b0a052bf7b9 ======LOGS====== Aug 22 18:53:18.611368343 (337h27m13.042448474s) [upload=351066]: Starting upload Aug 22 18:53:18.611414451 (337h27m13.042494582s) [upload=351066]: Done starting upload Aug 22 18:53:18.611493242 (337h27m13.042573383s) [upload=351066]: Uploading segments... Aug 22 18:53:18.611592071 (337h27m13.042672202s) [upload=351066]: Got next segment. Inline: false Aug 22 18:53:18.611620335 (337h27m13.042700466s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:53:18.611624873 (337h27m13.042705004s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:53:19.067533922 (337h27m13.498614053s) [seg_pos={0 0}, upload=351066]: Begin upload segment... Aug 22 18:53:19.067578687 (337h27m13.498658818s) [seg_pos={0 0}, upload=351066]: Done begin upload segment. Aug 22 18:53:19.067598065 (337h27m13.498678196s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.067955498 (337h27m13.499035629s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068223632 (337h27m13.499303763s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068535899 (337h27m13.49961603s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.068796498 (337h27m13.499876619s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.069036867 (337h27m13.500116998s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070079632 (337h27m13.501159763s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070493725 (337h27m13.501573847s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.070784561 (337h27m13.501864692s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071097190 (337h27m13.502177321s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071381353 (337h27m13.502461485s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071623136 (337h27m13.502703267s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.071868736 (337h27m13.502948867s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072111000 (337h27m13.503191131s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072363032 (337h27m13.503443163s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072631386 (337h27m13.503711517s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.072872667 (337h27m13.503952798s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:19.073093810 (337h27m13.504173931s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Uploading piece... Aug 22 18:53:20.595594732 (337h27m15.026674853s) [upload=351066]: Next returned nil segment Aug 22 18:53:20.595612135 (337h27m15.026692266s) [upload=351066]: Waiting for error group managing segments... Aug 22 18:53:21.007836407 (337h27m15.438916578s) [noise=true , address=45.91.116.7:4015 , piece_id=YZSWC4FJSVP4VZDTEYQNQRH25M4PHVWDBUAGGTBUF45V35UTDGAQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.025911572 (337h27m15.456991703s) [noise=true , address=77.90.31.18:4824 , piece_id=Y2EYNTGEOY5U2COYH33ZCZJOZVHNMLGASPXMF4RO5B65BGUFPMLA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.028467853 (337h27m15.459547984s) [noise=true , address=104.36.82.7:4925 , piece_id=HLGVQHZQA23IA4L6JODRIRT67ULKQ733G7MCFZ7SE6S4IDDIFVRQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.047663913 (337h27m15.478744044s) [noise=true , address=77.90.31.16:4742 , piece_id=PHHMRPHPN54X5VLPPVCGETEJVEVLGWXEIBEB6RJRNJPN2R6BQRGQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.053450645 (337h27m15.484530786s) [noise=true , address=104.36.82.17:4337 , piece_id=BIXBXZYI2J3PCLSQ4VATBWUPMENUTB3GLESYU6YB3WBCX2JB5CVQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.069484074 (337h27m15.500564205s) [noise=true , address=104.36.82.50:4013 , piece_id=AQ5ZROYXXXZGKW3LZOU6LXK5VWMLAPNMEVRCUATFJKSPWOAXXCRA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.096650765 (337h27m15.527730896s) [noise=true , address=104.36.82.21:4477 , piece_id=7RFIUHMUFOA35S3CC73OTNWHXZYIFIBIEHXJSZ7YKQYFGF6JXK2A, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.105896205 (337h27m15.536976336s) [noise=true , address=104.36.82.45:4909 , piece_id=T6FEZSE6CSUR4F7ZFFX3L56IJEMRLTHVXMQWQAXSP3AYIFWTMIFQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.126992952 (337h27m15.558073083s) [noise=true , address=77.90.31.10:4547 , piece_id=3CARVLBRAMOVAFNEEGHAW5ADXTYWTUM663XA56UASJPC6VEECXWQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.181061637 (337h27m15.612141768s) [noise=true , address=77.90.31.21:4908 , piece_id=BOKGMAE4BX6XCEP2L4PE7YJYKKNP6RYX3TJ4HH52IQ7FDMOITRSA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.192320048 (337h27m15.623400179s) [noise=true , address=104.36.82.12:4169 , piece_id=T6Y5FF36EC5WVB6MGY3LAEEWTXIBFXINNOYSW5BTHJDXYQOS2S6Q, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.196002525 (337h27m15.627082656s) [noise=true , address=104.36.82.43:4853 , piece_id=QKVD4M3BOWZL7JULCWNHVAGIMMJZM4KDPTRKCTB6WGBPFRJ7VFZQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.244578892 (337h27m15.675659023s) [noise=true , address=104.36.82.2:4764 , piece_id=SOJP6VZLZQJ2OFGMJ3A6N4TORHHTO2MOZG3W2ZVZBM5O7M2H2MJQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317255260 (337h27m15.748335391s) [noise=true , address=104.36.82.47:4535 , piece_id=V5VXV2TJABNXM55HKIC76MMOEM6NN2TRAYONQ675FTACQTWHM4VA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: <nil> Aug 22 18:53:21.317286951 (337h27m15.748367072s) [seg_pos={0 0}, upload=351066]: Segment reached optimal threshold of 14 pieces. Aug 22 18:53:21.317387413 (337h27m15.748467544s) [noise=true , address=104.36.82.42:4821 , piece_id=YBRJAMAFT2RPMAB46YWGWBWGLYULNU3Q6FHNJ2X7USUCIS3WJVBQ, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317447548 (337h27m15.748527679s) [noise=true , address=104.36.82.11:4116 , piece_id=DGSKZRGUH2DQOOB3YKDJEEGX3EYEP5SIG2WYWFVBRL3BTKT3BXTA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123aoo9g11aJP9hiL8shu2dx44tjyKzEWqPUjHUdXtR7LLFyng8): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317498345 (337h27m15.748578466s) [noise=true , address=104.36.82.16:4317 , piece_id=AY7F6UJBVEXWT35PSP7WQBLRTNQA7VXW2WUAVMKTU5HNXNOOS7QA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 22 18:53:21.317580633 (337h27m15.748660764s) [noise=true , address=45.91.116.12:4239 , piece_id=A433QYTI7FU3FANL3TKUL52UBD6KLMXBJ2WIPSSBBNRBV5YXJKJA, seg_pos={0 0}, upload=351066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DdwuGVzn339aKBRre1NvC9rUwSbL8v4GLwWj5TZBRP6RvLDnF): protocol: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35012->45.91.116.12:4239: use of closed network connection Aug 22 18:53:21.317604088 (337h27m15.748684219s) [seg_pos={0 0}, upload=351066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:53:21.317678330 (337h27m15.748758461s) [upload=351066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:53:21.317682508 (337h27m15.748762649s) [upload=351066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:53:21.615966306 (337h27m16.047046437s) [upload=351066]: Done uploading segments... Aug 22 18:53:21.615987125 (337h27m16.047067306s) [upload=351066]: Upload finished. err: <nil> Duration: 5.983768501s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X...X.X.................X.............X...........X........] duration: 5.917589846s FAIL (2025-08-22 18:53:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 5.917615114s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X.....................................X..............] duration: 2.087684596s FAIL (2025-08-22 18:52:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.087714833s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X.....................................X.............] duration: 2.01792756s FAIL (2025-08-22 18:52:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.017973287s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.972178327s FAIL (2025-08-22 18:52:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.972199388s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X...X.X.................X.............X...........X.......] duration: 6.124701395s FAIL (2025-08-22 18:52:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 6.124729929s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X.....................................X............] duration: 2.188718397s FAIL (2025-08-22 18:51:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.18873587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X...X.X.................X.............X...........X......] duration: 6.923581798s FAIL (2025-08-22 18:51:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 6.92361475s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.226368898s FAIL (2025-08-22 18:51:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.226393144s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X.....................................X...........] duration: 2.276114751s FAIL (2025-08-22 18:50:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.276146331s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.091688372s FAIL (2025-08-22 18:50:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.091708369s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X.....................................X..........] duration: 2.097792027s FAIL (2025-08-22 18:50:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.097821984s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.15511376s FAIL (2025-08-22 18:50:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.155134109s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X...X.X.................X.............X...........X.....] duration: 5.814893868s FAIL (2025-08-22 18:50:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 5.814925617s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X.....................................X.........] duration: 1.903919847s FAIL (2025-08-22 18:49:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 1.903939726s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.984930865s FAIL (2025-08-22 18:49:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.984952466s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X.....................................X........] duration: 2.153247946s FAIL (2025-08-22 18:49:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.153268204s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X...X.X.................X.............X...........X....] duration: 5.992027255s FAIL (2025-08-22 18:49:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 5.992057604s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.196850901s FAIL (2025-08-22 18:48:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.196874857s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X.....................................X.......] duration: 2.123130019s FAIL (2025-08-22 18:48:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.123159866s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.916193267s FAIL (2025-08-22 18:48:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.916218225s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X.....................................X......] duration: 2.107275747s FAIL (2025-08-22 18:48:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.107300375s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X...X.X.................X.............X...........X...] duration: 6.024455902s FAIL (2025-08-22 18:47:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 6.024478705s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.84121073s FAIL (2025-08-22 18:47:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.841233082s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X...............X.....................................X.....] duration: 1.855732199s FAIL (2025-08-22 18:47:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 1.855762947s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.822945464s FAIL (2025-08-22 18:47:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.822968949s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X...X.X.................X.............X...........X..] duration: 5.882201963s FAIL (2025-08-22 18:46:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 5.882229446s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X...............X.....................................X....] duration: 1.98739652s FAIL (2025-08-22 18:46:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 1.987427519s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X...............X.....................................X...] duration: 2.080460209s FAIL (2025-08-22 18:46:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.08048622s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.08731011s FAIL (2025-08-22 18:46:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.087330328s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X...X.X.................X.............X...........X.] duration: 6.065706978s FAIL (2025-08-22 18:46:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 6.065737828s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X...............X.....................................X..] duration: 2.083571231s FAIL (2025-08-22 18:45:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 2.083592883s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.887535604s FAIL (2025-08-22 18:45:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.887635306s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X...X.X.................X.............X...........X] duration: 5.882231834s FAIL (2025-08-22 18:45:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 270fc94d48df789353e597d627060cff840c17fca032d0191482d98316e3268a != before 0aa322ea74c48f5f202a85a45358b293964c6e9659cb7d972de73c992f6c8561 ======LOGS====== Aug 22 18:45:00.427186443 (337h18m54.858266614s) [upload=350918]: Starting upload Aug 22 18:45:00.427280534 (337h18m54.858360695s) [upload=350918]: Done starting upload Aug 22 18:45:00.427368602 (337h18m54.858448723s) [upload=350918]: Uploading segments... Aug 22 18:45:00.427672483 (337h18m54.858752664s) [upload=350918]: Got next segment. Inline: false Aug 22 18:45:00.427708011 (337h18m54.858788142s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:45:00.427714093 (337h18m54.858794224s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:45:00.898588993 (337h18m55.329669124s) [seg_pos={0 0}, upload=350918]: Begin upload segment... Aug 22 18:45:00.898631894 (337h18m55.329712025s) [seg_pos={0 0}, upload=350918]: Done begin upload segment. Aug 22 18:45:00.898655099 (337h18m55.32973522s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899021781 (337h18m55.330101922s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899287699 (337h18m55.33036783s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899545153 (337h18m55.330625284s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.899827322 (337h18m55.330907453s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900093902 (337h18m55.331174034s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900306209 (337h18m55.33138634s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900556388 (337h18m55.331636519s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900763895 (337h18m55.331844026s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.900997441 (337h18m55.332077562s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901192966 (337h18m55.332273097s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901378631 (337h18m55.332458762s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901580408 (337h18m55.332660529s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901766113 (337h18m55.332846244s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.901833902 (337h18m55.332914033s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902060295 (337h18m55.333140436s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902442277 (337h18m55.333522408s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:00.902817194 (337h18m55.333897325s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Uploading piece... Aug 22 18:45:02.398652987 (337h18m56.829733118s) [upload=350918]: Next returned nil segment Aug 22 18:45:02.398678005 (337h18m56.829758136s) [upload=350918]: Waiting for error group managing segments... Aug 22 18:45:02.812133228 (337h18m57.243213359s) [noise=true , address=77.90.31.16:4767 , piece_id=KR67NIVEKOWL7WGZ44S22YIQUUODPZBHQVLZDIAG22ODF3Y6A4UA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.829491057 (337h18m57.260571188s) [noise=true , address=104.36.82.17:4331 , piece_id=7JO2AEY6NDMFYIOIWKDWUDSLPDQL7TQ5QZMZPO62FSXNACWFIBZA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.836190537 (337h18m57.267270668s) [noise=true , address=104.36.82.38:4714 , piece_id=KOD5MH7XCLP26WECVB4S7J2I4ZQVZTVXVPECPNUJWGDMSTDO7XEQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.849759906 (337h18m57.280840047s) [noise=true , address=104.36.82.22:4515 , piece_id=GU4VYJRBZ57HQJHV4C4LVI2QD2CNU2ZZMPZHOOO3JYVYK45HTO5A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.909092203 (337h18m57.340172344s) [noise=true , address=104.36.82.60:4371 , piece_id=VIGFUEGO3OU64HOSIRZA6DSPQUUUJBTRH2M4F74H5SQZOJD4U4AQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.932882901 (337h18m57.363963032s) [noise=true , address=45.91.116.4:4957 , piece_id=2OVR5E6X75Z7Z3WVUQUFDAJQAAJ4EQDMHTDSH6KRVLVZVFSCHFEA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.942897477 (337h18m57.373977618s) [noise=true , address=104.36.82.41:4774 , piece_id=XXHRI4LHYG23ACNDLVAUH2RZAZWESA6DRHD4GVBPAJLU2T7KEHIA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.971208216 (337h18m57.402288357s) [noise=true , address=104.36.82.45:4907 , piece_id=QVQ62KHUJFOROAE5N7ZKFOXH7FWRQ4UTXW4CGEXE2YQ72X4WWWNA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.977382552 (337h18m57.408462693s) [noise=true , address=104.36.82.46:4501 , piece_id=5UB3F7PZDIMIKYPOKKNSTDCBH4VMSB5DBSPNMJCOD665RSGX7YFQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.981932268 (337h18m57.413012409s) [noise=true , address=104.36.82.7:4935 , piece_id=3DPWJWLLKLRDNCPT4YRBXFIS54JQFG37QPO7TREB3EPV3SA22QHQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.990711741 (337h18m57.421791882s) [noise=true , address=104.36.82.39:4754 , piece_id=D3Z257VSZHKEXYI7GZBENFWDCAMAJMVKXMZS5IA6UPNZ7YFFJF7A, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:02.994441789 (337h18m57.42552193s) [noise=true , address=77.90.31.10:4551 , piece_id=FEDC7TEGJ2NKF4VK2EGH2KL6PEETAIF2C6TCQWP6L5P42HLGLRTQ, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.001220321 (337h18m57.432300472s) [noise=true , address=104.36.82.3:4784 , piece_id=RRW3Z4MAKXQK4FTVP2TQJYVKVSRSYAJG23TQRAYRGSH44Z372ELA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026009651 (337h18m57.457089782s) [noise=true , address=104.36.82.50:4016 , piece_id=F5BLI6TCTXJYJE3G3735ZBXCWTU6AXTTZJDJLXKXY7G4GCNVUJCA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: <nil> Aug 22 18:45:03.026042765 (337h18m57.457122906s) [seg_pos={0 0}, upload=350918]: Segment reached optimal threshold of 14 pieces. Aug 22 18:45:03.026255291 (337h18m57.457335422s) [noise=true , address=77.90.31.14:4676 , piece_id=N45ODLBGDRP43XEQENQBLUT5PONGQCWQRJKU36ZIE652NV2OF2GA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tpRh8Xvyxe72LDgR9rkUFx9qV88axb9DUYDhRnrzST2cQuHqC): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026337758 (337h18m57.4574179s) [noise=true , address=77.90.31.15:4701 , piece_id=WM2VAXY72PHWXHPXE7LG4GWHW6XHIEOMEESHGKY5ZMUXDAZ2CGVA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026431368 (337h18m57.457511499s) [noise=true , address=77.90.31.13:4652 , piece_id=PCJCMGXXAJA7ZCWN4T2E7LIU7O6TMZKFUGQADT2KGOUMTU6JXJ7Q, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026509147 (337h18m57.457589288s) [noise=true , address=45.91.116.12:4248 , piece_id=MNORXXINAJWFJ7J7J74ZGDH6Q4OPH2SN7SNWDYP4GUNOOLCVPEMA, seg_pos={0 0}, upload=350918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZS8eHtnmixkxuGXZrV7KQYXuKzTm6hejtfMBCpMZctn9hceKG): protocol: expected piece hash; context canceled; EOF Aug 22 18:45:03.026521029 (337h18m57.45760116s) [seg_pos={0 0}, upload=350918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:45:03.026600031 (337h18m57.457680162s) [upload=350918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:45:03.026604599 (337h18m57.45768474s) [upload=350918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:45:03.324950004 (337h18m57.756030145s) [upload=350918]: Done uploading segments... Aug 22 18:45:03.324975863 (337h18m57.756056004s) [upload=350918]: Upload finished. err: <nil> Duration: 5.882261368s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X...............X.....................................X.] duration: 1.85580449s FAIL (2025-08-22 18:44:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 1.855827202s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X...X.X.................X.............X...........] duration: 5.842233538s FAIL (2025-08-22 18:44:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 22 18:34:43.453438184 (337h8m37.884518305s) [upload=350752]: Starting upload Aug 22 18:34:43.453491707 (337h8m37.884571838s) [upload=350752]: Done starting upload Aug 22 18:34:43.453567491 (337h8m37.884647622s) [upload=350752]: Uploading segments... Aug 22 18:34:43.453677812 (337h8m37.884757943s) [upload=350752]: Got next segment. Inline: false Aug 22 18:34:43.453708861 (337h8m37.884788992s) [upload=350752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:34:43.453713211 (337h8m37.884793342s) [upload=350752]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:34:43.910518285 (337h8m38.341598416s) [seg_pos={0 0}, upload=350752]: Begin upload segment... Aug 22 18:34:43.910567760 (337h8m38.341647891s) [seg_pos={0 0}, upload=350752]: Done begin upload segment. Aug 22 18:34:43.910580785 (337h8m38.341660916s) [noise=true , address=104.36.82.55:4213 , piece_id=EWCVGA4VX3T2UWXPMT44OL5FM7JQJP55WKKRX67UOFSN6RXCSDXQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.910588891 (337h8m38.341669022s) [noise=true , address=104.36.82.7:4917 , piece_id=QL3BMLYMRBEYB2COKGODZO5O4ZT7WA7AYRSLWEZWJ7CE7GPEJ4QQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.910809182 (337h8m38.341889313s) [noise=true , address=45.91.116.10:4107 , piece_id=LG5XD5VLQKLJX7ZCIIXKI2JLTV57JZJTXUUY2WEQMG4ATYFKX53A, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911048529 (337h8m38.34212866s) [noise=true , address=104.36.82.13:4206 , piece_id=EBRQDST2WAZM4OCSGTYEI4MCHMV6CJ5CKB5FCV4ND7OWKB5CZLKA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911321923 (337h8m38.342402054s) [noise=true , address=104.36.82.34:4574 , piece_id=US3OIJEVAP7RYJ2Y4ZPMB7IIGSJJ3VYRP6IMSWZR2MFLHMG2EKMA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911579025 (337h8m38.342659156s) [noise=true , address=104.36.82.23:4449 , piece_id=YFHGSYFJCH2X6MC3A25ONUQITG2F5LM7AB4ODNG5B3RNL4KHRUBQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911834413 (337h8m38.342914545s) [noise=true , address=104.36.82.5:4872 , piece_id=XC7QFSIRS6L4U2XXH6XGJOFTB3YOYOSP4YYSW4XIGUIG4SHDF6EA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912084201 (337h8m38.343164332s) [noise=true , address=79.127.220.98:4427 , piece_id=IRG7COZMXE7BJGL5FYLDEFPX7MNMPIYT5SUZNG4QFQC5HIPJ2GTQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912384326 (337h8m38.343464457s) [noise=true , address=104.36.82.14:4233 , piece_id=HXRACJPJQVIFEAJYLQJQ3OND5VIY623B7G2TFRZBLH227SPFVFZQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912596271 (337h8m38.343676392s) [noise=true , address=104.36.82.60:4345 , piece_id=P62ONB2XZMFRBKR3BWVBQZCJIK2GA3WBTRUIL6XSBKK3OVZL3D4A, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912789471 (337h8m38.343869592s) [noise=true , address=77.90.31.17:4783 , piece_id=P6XUR6YNSETWZ4PQ75G3U7G434ZNSHVMNOMKGH62A3DGTATDQPAQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912982060 (337h8m38.344062191s) [noise=true , address=104.36.82.49:4993 , piece_id=HH32R3VVV2SQQK3ZTW7XBRPS2R4356YVNYEC3XXRW3LHFSTNU4WA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913185979 (337h8m38.34426611s) [noise=true , address=104.36.82.12:4161 , piece_id=7UV7WCDWS74AMZRO4MDH6JLT5A447R4ZZXEUALI6DEQMWFYK3X2Q, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913377816 (337h8m38.344457947s) [noise=true , address=45.91.116.11:4141 , piece_id=QDOGCZ7F2MHS4H5NTDJC4UGH5LDRV7AIWH2EQ5NKECI2KJCF7M7A, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913559033 (337h8m38.344639155s) [noise=true , address=77.90.31.18:4827 , piece_id=G7YKRDXWTHR2OKYC526ZULB5TPJLYW5YMM5C3XUV6ZT4D3CY4DLQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913756681 (337h8m38.344836812s) [noise=true , address=45.91.116.5:4328 , piece_id=GDRSZ4F4Q2I2VQCBGM3GNSEOBTMYSAKXD5G7ZBPLQA5TO37BSOSA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913960402 (337h8m38.345040533s) [noise=true , address=79.127.220.97:4377 , piece_id=2UOABAAJKZNW3W55F2RENTZOTYMU32IR7VCNAMXSGALKAJOEBSQQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.914151447 (337h8m38.345231578s) [noise=true , address=104.36.82.11:4125 , piece_id=6NXXOWFCNTK2WWMNRXTFL3R2A5HOLRO7DQ46IUQ57D6IOXLH2CJA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:45.389715828 (337h8m39.820795959s) [upload=350752]: Next returned nil segment Aug 22 18:34:45.389732079 (337h8m39.82081221s) [upload=350752]: Waiting for error group managing segments... Aug 22 18:34:45.775177214 (337h8m40.206257345s) [noise=true , address=79.127.220.97:4377 , piece_id=2UOABAAJKZNW3W55F2RENTZOTYMU32IR7VCNAMXSGALKAJOEBSQQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.881108404 (337h8m40.312188535s) [noise=true , address=104.36.82.55:4213 , piece_id=EWCVGA4VX3T2UWXPMT44OL5FM7JQJP55WKKRX67UOFSN6RXCSDXQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.890602780 (337h8m40.321682911s) [noise=true , address=45.91.116.5:4328 , piece_id=GDRSZ4F4Q2I2VQCBGM3GNSEOBTMYSAKXD5G7ZBPLQA5TO37BSOSA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.894637731 (337h8m40.325717852s) [noise=true , address=77.90.31.17:4783 , piece_id=P6XUR6YNSETWZ4PQ75G3U7G434ZNSHVMNOMKGH62A3DGTATDQPAQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.906915343 (337h8m40.337995474s) [noise=true , address=104.36.82.12:4161 , piece_id=7UV7WCDWS74AMZRO4MDH6JLT5A447R4ZZXEUALI6DEQMWFYK3X2Q, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.909015832 (337h8m40.340095963s) [noise=true , address=45.91.116.11:4141 , piece_id=QDOGCZ7F2MHS4H5NTDJC4UGH5LDRV7AIWH2EQ5NKECI2KJCF7M7A, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.909315476 (337h8m40.340395607s) [noise=true , address=45.91.116.10:4107 , piece_id=LG5XD5VLQKLJX7ZCIIXKI2JLTV57JZJTXUUY2WEQMG4ATYFKX53A, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.920405655 (337h8m40.351485786s) [noise=true , address=104.36.82.49:4993 , piece_id=HH32R3VVV2SQQK3ZTW7XBRPS2R4356YVNYEC3XXRW3LHFSTNU4WA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.945707198 (337h8m40.376787329s) [noise=true , address=104.36.82.60:4345 , piece_id=P62ONB2XZMFRBKR3BWVBQZCJIK2GA3WBTRUIL6XSBKK3OVZL3D4A, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.979737151 (337h8m40.410817272s) [noise=true , address=104.36.82.5:4872 , piece_id=XC7QFSIRS6L4U2XXH6XGJOFTB3YOYOSP4YYSW4XIGUIG4SHDF6EA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.983752384 (337h8m40.414832515s) [noise=true , address=104.36.82.23:4449 , piece_id=YFHGSYFJCH2X6MC3A25ONUQITG2F5LM7AB4ODNG5B3RNL4KHRUBQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.041000462 (337h8m40.472080594s) [noise=true , address=104.36.82.13:4206 , piece_id=EBRQDST2WAZM4OCSGTYEI4MCHMV6CJ5CKB5FCV4ND7OWKB5CZLKA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.046744023 (337h8m40.477824144s) [noise=true , address=104.36.82.34:4574 , piece_id=US3OIJEVAP7RYJ2Y4ZPMB7IIGSJJ3VYRP6IMSWZR2MFLHMG2EKMA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.061602673 (337h8m40.492682804s) [noise=true , address=77.90.31.18:4827 , piece_id=G7YKRDXWTHR2OKYC526ZULB5TPJLYW5YMM5C3XUV6ZT4D3CY4DLQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.061627040 (337h8m40.492707171s) [seg_pos={0 0}, upload=350752]: Segment reached optimal threshold of 14 pieces. Aug 22 18:34:46.061751698 (337h8m40.492831829s) [noise=true , address=104.36.82.11:4125 , piece_id=6NXXOWFCNTK2WWMNRXTFL3R2A5HOLRO7DQ46IUQ57D6IOXLH2CJA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.061885955 (337h8m40.492966116s) [noise=true , address=104.36.82.7:4917 , piece_id=QL3BMLYMRBEYB2COKGODZO5O4ZT7WA7AYRSLWEZWJ7CE7GPEJ4QQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VaqXNoBYjwbbE9xhAoLzDuxZMdBPSm2aMw7bYm4qZqiEGp5CV): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.061909621 (337h8m40.492989752s) [noise=true , address=104.36.82.14:4233 , piece_id=HXRACJPJQVIFEAJYLQJQ3OND5VIY623B7G2TFRZBLH227SPFVFZQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124wXnmRuxo3o7AVwdSbUmai3SebaMV5qgScGVFSWFKSRvBNTyB): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.061984163 (337h8m40.493064294s) [noise=true , address=79.127.220.98:4427 , piece_id=IRG7COZMXE7BJGL5FYLDEFPX7MNMPIYT5SUZNG4QFQC5HIPJ2GTQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dfnafsytLN1SDZVjxhGSoLxwduLPEEZC3CdsFGSZyELiZisVq): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.062004032 (337h8m40.493084163s) [seg_pos={0 0}, upload=350752]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:34:46.062065699 (337h8m40.493145831s) [upload=350752]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:34:46.062071410 (337h8m40.493151531s) [upload=350752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:34:46.406094027 (337h8m40.837174158s) [upload=350752]: Done uploading segments... Aug 22 18:34:46.406112312 (337h8m40.837192443s) [upload=350752]: Upload finished. err: <nil> Duration: 5.84226562s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.815492592s FAIL (2025-08-22 18:44:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.81551767s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.117849091s FAIL (2025-08-22 18:43:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.117870284s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..........X...X.X.................X.............X..........] duration: 5.698791182s FAIL (2025-08-22 18:43:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 22 18:34:43.453438184 (337h8m37.884518305s) [upload=350752]: Starting upload Aug 22 18:34:43.453491707 (337h8m37.884571838s) [upload=350752]: Done starting upload Aug 22 18:34:43.453567491 (337h8m37.884647622s) [upload=350752]: Uploading segments... Aug 22 18:34:43.453677812 (337h8m37.884757943s) [upload=350752]: Got next segment. Inline: false Aug 22 18:34:43.453708861 (337h8m37.884788992s) [upload=350752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:34:43.453713211 (337h8m37.884793342s) [upload=350752]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 22 18:34:43.910518285 (337h8m38.341598416s) [seg_pos={0 0}, upload=350752]: Begin upload segment... Aug 22 18:34:43.910567760 (337h8m38.341647891s) [seg_pos={0 0}, upload=350752]: Done begin upload segment. Aug 22 18:34:43.910580785 (337h8m38.341660916s) [noise=true , address=104.36.82.55:4213 , piece_id=EWCVGA4VX3T2UWXPMT44OL5FM7JQJP55WKKRX67UOFSN6RXCSDXQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.910588891 (337h8m38.341669022s) [noise=true , address=104.36.82.7:4917 , piece_id=QL3BMLYMRBEYB2COKGODZO5O4ZT7WA7AYRSLWEZWJ7CE7GPEJ4QQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.910809182 (337h8m38.341889313s) [noise=true , address=45.91.116.10:4107 , piece_id=LG5XD5VLQKLJX7ZCIIXKI2JLTV57JZJTXUUY2WEQMG4ATYFKX53A, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911048529 (337h8m38.34212866s) [noise=true , address=104.36.82.13:4206 , piece_id=EBRQDST2WAZM4OCSGTYEI4MCHMV6CJ5CKB5FCV4ND7OWKB5CZLKA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911321923 (337h8m38.342402054s) [noise=true , address=104.36.82.34:4574 , piece_id=US3OIJEVAP7RYJ2Y4ZPMB7IIGSJJ3VYRP6IMSWZR2MFLHMG2EKMA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911579025 (337h8m38.342659156s) [noise=true , address=104.36.82.23:4449 , piece_id=YFHGSYFJCH2X6MC3A25ONUQITG2F5LM7AB4ODNG5B3RNL4KHRUBQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.911834413 (337h8m38.342914545s) [noise=true , address=104.36.82.5:4872 , piece_id=XC7QFSIRS6L4U2XXH6XGJOFTB3YOYOSP4YYSW4XIGUIG4SHDF6EA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912084201 (337h8m38.343164332s) [noise=true , address=79.127.220.98:4427 , piece_id=IRG7COZMXE7BJGL5FYLDEFPX7MNMPIYT5SUZNG4QFQC5HIPJ2GTQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912384326 (337h8m38.343464457s) [noise=true , address=104.36.82.14:4233 , piece_id=HXRACJPJQVIFEAJYLQJQ3OND5VIY623B7G2TFRZBLH227SPFVFZQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912596271 (337h8m38.343676392s) [noise=true , address=104.36.82.60:4345 , piece_id=P62ONB2XZMFRBKR3BWVBQZCJIK2GA3WBTRUIL6XSBKK3OVZL3D4A, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912789471 (337h8m38.343869592s) [noise=true , address=77.90.31.17:4783 , piece_id=P6XUR6YNSETWZ4PQ75G3U7G434ZNSHVMNOMKGH62A3DGTATDQPAQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.912982060 (337h8m38.344062191s) [noise=true , address=104.36.82.49:4993 , piece_id=HH32R3VVV2SQQK3ZTW7XBRPS2R4356YVNYEC3XXRW3LHFSTNU4WA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913185979 (337h8m38.34426611s) [noise=true , address=104.36.82.12:4161 , piece_id=7UV7WCDWS74AMZRO4MDH6JLT5A447R4ZZXEUALI6DEQMWFYK3X2Q, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913377816 (337h8m38.344457947s) [noise=true , address=45.91.116.11:4141 , piece_id=QDOGCZ7F2MHS4H5NTDJC4UGH5LDRV7AIWH2EQ5NKECI2KJCF7M7A, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913559033 (337h8m38.344639155s) [noise=true , address=77.90.31.18:4827 , piece_id=G7YKRDXWTHR2OKYC526ZULB5TPJLYW5YMM5C3XUV6ZT4D3CY4DLQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913756681 (337h8m38.344836812s) [noise=true , address=45.91.116.5:4328 , piece_id=GDRSZ4F4Q2I2VQCBGM3GNSEOBTMYSAKXD5G7ZBPLQA5TO37BSOSA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.913960402 (337h8m38.345040533s) [noise=true , address=79.127.220.97:4377 , piece_id=2UOABAAJKZNW3W55F2RENTZOTYMU32IR7VCNAMXSGALKAJOEBSQQ, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:43.914151447 (337h8m38.345231578s) [noise=true , address=104.36.82.11:4125 , piece_id=6NXXOWFCNTK2WWMNRXTFL3R2A5HOLRO7DQ46IUQ57D6IOXLH2CJA, seg_pos={0 0}, upload=350752]: Uploading piece... Aug 22 18:34:45.389715828 (337h8m39.820795959s) [upload=350752]: Next returned nil segment Aug 22 18:34:45.389732079 (337h8m39.82081221s) [upload=350752]: Waiting for error group managing segments... Aug 22 18:34:45.775177214 (337h8m40.206257345s) [noise=true , address=79.127.220.97:4377 , piece_id=2UOABAAJKZNW3W55F2RENTZOTYMU32IR7VCNAMXSGALKAJOEBSQQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.881108404 (337h8m40.312188535s) [noise=true , address=104.36.82.55:4213 , piece_id=EWCVGA4VX3T2UWXPMT44OL5FM7JQJP55WKKRX67UOFSN6RXCSDXQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.890602780 (337h8m40.321682911s) [noise=true , address=45.91.116.5:4328 , piece_id=GDRSZ4F4Q2I2VQCBGM3GNSEOBTMYSAKXD5G7ZBPLQA5TO37BSOSA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.894637731 (337h8m40.325717852s) [noise=true , address=77.90.31.17:4783 , piece_id=P6XUR6YNSETWZ4PQ75G3U7G434ZNSHVMNOMKGH62A3DGTATDQPAQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.906915343 (337h8m40.337995474s) [noise=true , address=104.36.82.12:4161 , piece_id=7UV7WCDWS74AMZRO4MDH6JLT5A447R4ZZXEUALI6DEQMWFYK3X2Q, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.909015832 (337h8m40.340095963s) [noise=true , address=45.91.116.11:4141 , piece_id=QDOGCZ7F2MHS4H5NTDJC4UGH5LDRV7AIWH2EQ5NKECI2KJCF7M7A, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.909315476 (337h8m40.340395607s) [noise=true , address=45.91.116.10:4107 , piece_id=LG5XD5VLQKLJX7ZCIIXKI2JLTV57JZJTXUUY2WEQMG4ATYFKX53A, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.920405655 (337h8m40.351485786s) [noise=true , address=104.36.82.49:4993 , piece_id=HH32R3VVV2SQQK3ZTW7XBRPS2R4356YVNYEC3XXRW3LHFSTNU4WA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.945707198 (337h8m40.376787329s) [noise=true , address=104.36.82.60:4345 , piece_id=P62ONB2XZMFRBKR3BWVBQZCJIK2GA3WBTRUIL6XSBKK3OVZL3D4A, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.979737151 (337h8m40.410817272s) [noise=true , address=104.36.82.5:4872 , piece_id=XC7QFSIRS6L4U2XXH6XGJOFTB3YOYOSP4YYSW4XIGUIG4SHDF6EA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:45.983752384 (337h8m40.414832515s) [noise=true , address=104.36.82.23:4449 , piece_id=YFHGSYFJCH2X6MC3A25ONUQITG2F5LM7AB4ODNG5B3RNL4KHRUBQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.041000462 (337h8m40.472080594s) [noise=true , address=104.36.82.13:4206 , piece_id=EBRQDST2WAZM4OCSGTYEI4MCHMV6CJ5CKB5FCV4ND7OWKB5CZLKA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.046744023 (337h8m40.477824144s) [noise=true , address=104.36.82.34:4574 , piece_id=US3OIJEVAP7RYJ2Y4ZPMB7IIGSJJ3VYRP6IMSWZR2MFLHMG2EKMA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.061602673 (337h8m40.492682804s) [noise=true , address=77.90.31.18:4827 , piece_id=G7YKRDXWTHR2OKYC526ZULB5TPJLYW5YMM5C3XUV6ZT4D3CY4DLQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: <nil> Aug 22 18:34:46.061627040 (337h8m40.492707171s) [seg_pos={0 0}, upload=350752]: Segment reached optimal threshold of 14 pieces. Aug 22 18:34:46.061751698 (337h8m40.492831829s) [noise=true , address=104.36.82.11:4125 , piece_id=6NXXOWFCNTK2WWMNRXTFL3R2A5HOLRO7DQ46IUQ57D6IOXLH2CJA, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.061885955 (337h8m40.492966116s) [noise=true , address=104.36.82.7:4917 , piece_id=QL3BMLYMRBEYB2COKGODZO5O4ZT7WA7AYRSLWEZWJ7CE7GPEJ4QQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VaqXNoBYjwbbE9xhAoLzDuxZMdBPSm2aMw7bYm4qZqiEGp5CV): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.061909621 (337h8m40.492989752s) [noise=true , address=104.36.82.14:4233 , piece_id=HXRACJPJQVIFEAJYLQJQ3OND5VIY623B7G2TFRZBLH227SPFVFZQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124wXnmRuxo3o7AVwdSbUmai3SebaMV5qgScGVFSWFKSRvBNTyB): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.061984163 (337h8m40.493064294s) [noise=true , address=79.127.220.98:4427 , piece_id=IRG7COZMXE7BJGL5FYLDEFPX7MNMPIYT5SUZNG4QFQC5HIPJ2GTQ, seg_pos={0 0}, upload=350752]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dfnafsytLN1SDZVjxhGSoLxwduLPEEZC3CdsFGSZyELiZisVq): protocol: expected piece hash; context canceled; EOF Aug 22 18:34:46.062004032 (337h8m40.493084163s) [seg_pos={0 0}, upload=350752]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 22 18:34:46.062065699 (337h8m40.493145831s) [upload=350752]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 22 18:34:46.062071410 (337h8m40.493151531s) [upload=350752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:34:46.406094027 (337h8m40.837174158s) [upload=350752]: Done uploading segments... Aug 22 18:34:46.406112312 (337h8m40.837192443s) [upload=350752]: Upload finished. err: <nil> Duration: 5.698885933s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X...............X.....................................X] duration: 1.595403708s FAIL (2025-08-22 18:43:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 22 18:43:36.050571231 (337h17m30.481651362s) [upload=350895]: Starting upload Aug 22 18:43:36.050622920 (337h17m30.481703041s) [upload=350895]: Done starting upload Aug 22 18:43:36.050695740 (337h17m30.481775871s) [upload=350895]: Uploading segments... Aug 22 18:43:36.050797344 (337h17m30.481877475s) [upload=350895]: Got next segment. Inline: true Aug 22 18:43:36.050814246 (337h17m30.481894377s) [upload=350895]: Waiting for error group managing segments... Aug 22 18:43:36.050859153 (337h17m30.481939274s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 22 18:43:36.050864543 (337h17m30.481944674s) [upload=350895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 22 18:43:36.050867368 (337h17m30.481947499s) [upload=350895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 22 18:43:36.341871202 (337h17m30.772951333s) [upload=350895]: Done uploading segments... Aug 22 18:43:36.341888806 (337h17m30.772968927s) [upload=350895]: Upload finished. err: <nil> Duration:* 1.595427945s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.326317053s FAIL (2025-08-22 18:43:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.3263411s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)