Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..X...............X.....................X.......X...........] duration: 1.857699135s FAIL (2025-08-13 02:46:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.857724672s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X............X.................................] duration: 7.488776146s FAIL (2025-08-13 02:45:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.488803168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...X...............X.....................X.......X..........] duration: 1.848053738s FAIL (2025-08-13 02:45:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.848083645s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....X...............X.....................X.......X.........] duration: 1.901425366s FAIL (2025-08-13 02:45:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.901451857s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.695681475s FAIL (2025-08-13 02:45:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.695711151s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X............X................................] duration: 7.49614365s FAIL (2025-08-13 02:45:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.496169549s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X............X...............................] duration: 10.240234853s FAIL (2025-08-13 02:44:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.240256657s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.813304986s FAIL (2025-08-13 02:44:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.813331135s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....X...............X.....................X.......X........] duration: 1.872800447s FAIL (2025-08-13 02:44:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.872828501s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X............X..............................] duration: 7.453198814s FAIL (2025-08-13 02:44:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.453224012s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X............X.............................] duration: 6.457066797s FAIL (2025-08-13 02:43:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.457433838s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.04705492s FAIL (2025-08-13 02:43:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.047080009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X...............X.....................X.......X.......] duration: 1.890553955s FAIL (2025-08-13 02:43:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.890574364s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.804793481s FAIL (2025-08-13 02:43:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.804824611s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X............X............................] duration: 7.050833694s FAIL (2025-08-13 02:43:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.050858101s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.599640217s FAIL (2025-08-13 02:42:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.599688509s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X............X...........................] duration: 7.315198505s FAIL (2025-08-13 02:42:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.315225226s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X...............X.....................X.......X......] duration: 1.842680024s FAIL (2025-08-13 02:42:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.842710533s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.247274016s FAIL (2025-08-13 02:42:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.247321138s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X............X..........................] duration: 7.465125042s FAIL (2025-08-13 02:42:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.465147134s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X............X.........................] duration: 6.685112236s FAIL (2025-08-13 02:41:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.685134778s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.982307702s FAIL (2025-08-13 02:41:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.982361455s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X...............X.....................X.......X.....] duration: 1.852608612s FAIL (2025-08-13 02:41:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.852636775s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X............X........................] duration: 7.457373545s FAIL (2025-08-13 02:41:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.457409584s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.863566375s FAIL (2025-08-13 02:41:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.863594527s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X...............X.....................X.......X....] duration: 2.006262163s FAIL (2025-08-13 02:40:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.00629196s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X............X.......................] duration: 7.507567801s FAIL (2025-08-13 02:40:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.507633436s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.697639213s FAIL (2025-08-13 02:40:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.697673948s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X............X......................] duration: 7.215124349s FAIL (2025-08-13 02:40:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.215140691s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X...............X.....................X.......X...] duration: 2.135170119s FAIL (2025-08-13 02:39:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.135195307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X...............X.....................X.......X..] duration: 1.876380668s FAIL (2025-08-13 02:39:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.876401177s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X............X.....................] duration: 7.744201387s FAIL (2025-08-13 02:39:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.744212408s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................X................] duration: 6.356912373s FAIL (2025-08-13 02:38:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.35693702s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X...............X.....................X.......X.] duration: 1.865217669s FAIL (2025-08-13 02:38:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.865244129s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X............X....................] duration: 7.784342716s FAIL (2025-08-13 02:38:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.784366342s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X............X...................] duration: 7.557289582s FAIL (2025-08-13 02:38:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.557311655s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................................X...............] duration: 5.823243715s FAIL (2025-08-13 02:37:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.823268682s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X...............X.....................X.......X] duration: 2.175876387s FAIL (2025-08-13 02:37:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.175896965s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X............X..................] duration: 7.516823162s FAIL (2025-08-13 02:37:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.516900751s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X...............X.....................X.......] duration: 1.843219758s FAIL (2025-08-13 02:36:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.843249463s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................................X..............] duration: 5.590819836s FAIL (2025-08-13 02:36:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.590845655s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X............X.................] duration: 7.260232195s FAIL (2025-08-13 02:36:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.260253394s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X...............X.....................X......] duration: 1.851599111s FAIL (2025-08-13 02:36:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.851626574s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X............X................] duration: 7.831409109s FAIL (2025-08-13 02:35:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.8314304s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................................X.............] duration: 5.853261407s FAIL (2025-08-13 02:35:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.853289572s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X............X...............] duration: 7.368433221s FAIL (2025-08-13 02:35:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.368464771s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X...............X.....................X.....] duration: 1.870376951s FAIL (2025-08-13 02:35:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.87040255s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................................X............] duration: 5.652573695s FAIL (2025-08-13 02:35:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.652599664s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X..........................................X...........] duration: 5.820713751s FAIL (2025-08-13 02:34:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.820742978s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X...............X.....................X....] duration: 1.87029237s FAIL (2025-08-13 02:34:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 13 02:31:46.519943428 (105h5m40.951023559s) [upload=109404]: Starting upload Aug 13 02:31:46.519991500 (105h5m40.951071631s) [upload=109404]: Done starting upload Aug 13 02:31:46.520038399 (105h5m40.95111853s) [upload=109404]: Uploading segments... Aug 13 02:31:46.520071203 (105h5m40.951151334s) [upload=109404]: Got next segment. Inline: true Aug 13 02:31:46.520075040 (105h5m40.951155171s) [upload=109404]: Waiting for error group managing segments... Aug 13 02:31:46.520106590 (105h5m40.951186721s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:31:46.520110868 (105h5m40.951190999s) [upload=109404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:31:46.520113272 (105h5m40.951193404s) [upload=109404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:31:46.824097443 (105h5m41.255177564s) [upload=109404]: Done uploading segments... Aug 13 02:31:46.824116239 (105h5m41.25519637s) [upload=109404]: Upload finished. err: <nil> Duration:* 1.870327106s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)