Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X............X.....................] duration: 7.744201387s FAIL (2025-08-13 02:39:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.744212408s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................X................] duration: 6.356912373s FAIL (2025-08-13 02:38:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.35693702s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X...............X.....................X.......X.] duration: 1.865217669s FAIL (2025-08-13 02:38:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.865244129s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X............X....................] duration: 7.784342716s FAIL (2025-08-13 02:38:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.784366342s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X............X...................] duration: 7.557289582s FAIL (2025-08-13 02:38:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.557311655s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................................X...............] duration: 5.823243715s FAIL (2025-08-13 02:37:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.823268682s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X...............X.....................X.......X] duration: 2.175876387s FAIL (2025-08-13 02:37:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.175896965s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X............X..................] duration: 7.516823162s FAIL (2025-08-13 02:37:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.516900751s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X...............X.....................X.......] duration: 1.843219758s FAIL (2025-08-13 02:36:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.843249463s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................................X..............] duration: 5.590819836s FAIL (2025-08-13 02:36:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.590845655s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X............X.................] duration: 7.260232195s FAIL (2025-08-13 02:36:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.260253394s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X...............X.....................X......] duration: 1.851599111s FAIL (2025-08-13 02:36:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.851626574s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X............X................] duration: 7.831409109s FAIL (2025-08-13 02:35:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.8314304s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................................X.............] duration: 5.853261407s FAIL (2025-08-13 02:35:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.853289572s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X............X...............] duration: 7.368433221s FAIL (2025-08-13 02:35:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.368464771s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X...............X.....................X.....] duration: 1.870376951s FAIL (2025-08-13 02:35:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.87040255s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................................X............] duration: 5.652573695s FAIL (2025-08-13 02:35:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.652599664s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X..........................................X...........] duration: 5.820713751s FAIL (2025-08-13 02:34:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.820742978s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X...............X.....................X....] duration: 1.87029237s FAIL (2025-08-13 02:34:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.870327106s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X..........................................X..........] duration: 5.840615944s FAIL (2025-08-13 02:34:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.840642995s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X............X..............] duration: 7.838003806s FAIL (2025-08-13 02:34:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.838031861s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X..........................................X.........] duration: 5.883537596s FAIL (2025-08-13 02:33:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.88356518s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X...............X.....................X...] duration: 2.057377272s FAIL (2025-08-13 02:33:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 2.05748048s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................X...............X.....................X..] duration: 1.956809895s FAIL (2025-08-13 02:33:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.956839883s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X..........................................X........] duration: 5.92477804s FAIL (2025-08-13 02:33:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.924846221s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X............X.............] duration: 6.738794839s FAIL (2025-08-13 02:33:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.738818744s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X..........................................X.......] duration: 5.963256773s FAIL (2025-08-13 02:32:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.963279076s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................X...............X.....................X.] duration: 1.857710682s FAIL (2025-08-13 02:32:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.857738514s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X............X............] duration: 7.566615939s FAIL (2025-08-13 02:32:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.566637982s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................X...............X.....................X] duration: 1.412041824s FAIL (2025-08-13 02:31:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.412074034s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X............X...........] duration: 6.52896146s FAIL (2025-08-13 02:31:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.52899846s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X..........................................X......] duration: 5.885916266s FAIL (2025-08-13 02:31:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.885946153s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X............X..........] duration: 7.06586952s FAIL (2025-08-13 02:31:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.065890901s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X..........................................X.....] duration: 6.196361408s FAIL (2025-08-13 02:30:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.196390062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X.....................X...............X.....................] duration: 1.906776883s FAIL (2025-08-13 02:30:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:12:40.260374501 (104h46m34.691454632s) [upload=109079]: Starting upload Aug 13 02:12:40.260426891 (104h46m34.691507022s) [upload=109079]: Done starting upload Aug 13 02:12:40.260474383 (104h46m34.691554514s) [upload=109079]: Uploading segments... Aug 13 02:12:40.260539667 (104h46m34.691619808s) [upload=109079]: Got next segment. Inline: true Aug 13 02:12:40.260546300 (104h46m34.691626431s) [upload=109079]: Waiting for error group managing segments... Aug 13 02:12:40.260578301 (104h46m34.691658432s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:12:40.260582178 (104h46m34.691662309s) [upload=109079]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:12:40.260584633 (104h46m34.691664765s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:12:40.572291201 (104h46m35.003371332s) [upload=109079]: Done uploading segments... Aug 13 02:12:40.572309206 (104h46m35.003389337s) [upload=109079]: Upload finished. err: <nil> Duration:* 1.906810537s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X............X.........] duration: 6.494846077s FAIL (2025-08-13 02:30:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.494869862s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X............X........] duration: 6.819073464s FAIL (2025-08-13 02:30:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.819096809s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X..........................................X....] duration: 6.059715957s FAIL (2025-08-13 02:30:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.059749611s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X............X.......] duration: 6.975971624s FAIL (2025-08-13 02:29:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.975997312s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X.....................X...............X....................] duration: 2.063388319s FAIL (2025-08-13 02:29:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:12:40.260374501 (104h46m34.691454632s) [upload=109079]: Starting upload Aug 13 02:12:40.260426891 (104h46m34.691507022s) [upload=109079]: Done starting upload Aug 13 02:12:40.260474383 (104h46m34.691554514s) [upload=109079]: Uploading segments... Aug 13 02:12:40.260539667 (104h46m34.691619808s) [upload=109079]: Got next segment. Inline: true Aug 13 02:12:40.260546300 (104h46m34.691626431s) [upload=109079]: Waiting for error group managing segments... Aug 13 02:12:40.260578301 (104h46m34.691658432s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:12:40.260582178 (104h46m34.691662309s) [upload=109079]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:12:40.260584633 (104h46m34.691664765s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:12:40.572291201 (104h46m35.003371332s) [upload=109079]: Done uploading segments... Aug 13 02:12:40.572309206 (104h46m35.003389337s) [upload=109079]: Upload finished. err: <nil> Duration:* 2.063413648s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X..........................................X...] duration: 6.061175613s FAIL (2025-08-13 02:29:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.061208055s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X.....................X...............X...................] duration: 1.92821485s FAIL (2025-08-13 02:29:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:12:40.260374501 (104h46m34.691454632s) [upload=109079]: Starting upload Aug 13 02:12:40.260426891 (104h46m34.691507022s) [upload=109079]: Done starting upload Aug 13 02:12:40.260474383 (104h46m34.691554514s) [upload=109079]: Uploading segments... Aug 13 02:12:40.260539667 (104h46m34.691619808s) [upload=109079]: Got next segment. Inline: true Aug 13 02:12:40.260546300 (104h46m34.691626431s) [upload=109079]: Waiting for error group managing segments... Aug 13 02:12:40.260578301 (104h46m34.691658432s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:12:40.260582178 (104h46m34.691662309s) [upload=109079]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:12:40.260584633 (104h46m34.691664765s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:12:40.572291201 (104h46m35.003371332s) [upload=109079]: Done uploading segments... Aug 13 02:12:40.572309206 (104h46m35.003389337s) [upload=109079]: Upload finished. err: <nil> Duration:* 1.928238213s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X............X......] duration: 6.480167911s FAIL (2025-08-13 02:29:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.480186195s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X............X.....] duration: 10.141379289s FAIL (2025-08-13 02:28:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.141401251s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X.....................X...............X..................] duration: 1.831546332s FAIL (2025-08-13 02:28:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:12:40.260374501 (104h46m34.691454632s) [upload=109079]: Starting upload Aug 13 02:12:40.260426891 (104h46m34.691507022s) [upload=109079]: Done starting upload Aug 13 02:12:40.260474383 (104h46m34.691554514s) [upload=109079]: Uploading segments... Aug 13 02:12:40.260539667 (104h46m34.691619808s) [upload=109079]: Got next segment. Inline: true Aug 13 02:12:40.260546300 (104h46m34.691626431s) [upload=109079]: Waiting for error group managing segments... Aug 13 02:12:40.260578301 (104h46m34.691658432s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:12:40.260582178 (104h46m34.691662309s) [upload=109079]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:12:40.260584633 (104h46m34.691664765s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:12:40.572291201 (104h46m35.003371332s) [upload=109079]: Done uploading segments... Aug 13 02:12:40.572309206 (104h46m35.003389337s) [upload=109079]: Upload finished. err: <nil> Duration:* 1.831569666s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X..........................................X..] duration: 6.629494161s FAIL (2025-08-13 02:28:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.629514599s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X............X....] duration: 23.833442732s FAIL (2025-08-13 02:28:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.833455948s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X.....................X...............X.................] duration: 1.856055254s FAIL (2025-08-13 02:27:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:12:40.260374501 (104h46m34.691454632s) [upload=109079]: Starting upload Aug 13 02:12:40.260426891 (104h46m34.691507022s) [upload=109079]: Done starting upload Aug 13 02:12:40.260474383 (104h46m34.691554514s) [upload=109079]: Uploading segments... Aug 13 02:12:40.260539667 (104h46m34.691619808s) [upload=109079]: Got next segment. Inline: true Aug 13 02:12:40.260546300 (104h46m34.691626431s) [upload=109079]: Waiting for error group managing segments... Aug 13 02:12:40.260578301 (104h46m34.691658432s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:12:40.260582178 (104h46m34.691662309s) [upload=109079]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:12:40.260584633 (104h46m34.691664765s) [upload=109079]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:12:40.572291201 (104h46m35.003371332s) [upload=109079]: Done uploading segments... Aug 13 02:12:40.572309206 (104h46m35.003389337s) [upload=109079]: Upload finished. err: <nil> Duration:* 1.856082858s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X..........................................X.] duration: 6.101213568s FAIL (2025-08-13 02:27:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.101239487s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X............X...] duration: 7.196931529s FAIL (2025-08-13 02:26:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.196951035s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)