Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X..XX.............................................] duration: 2.047797532s FAIL (2025-08-31 13:25:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.047820537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 6.676993095s FAIL (2025-08-31 13:25:48 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.677021378s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration: 774.065404ms target: https://us1.storj.io/login FAIL (2025-08-31 13:25:34 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 774.086905ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 6.448663625s FAIL (2025-08-31 13:25:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.44868722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 6.466122721s FAIL (2025-08-31 13:25:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.466167015s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 6.240766939s FAIL (2025-08-31 13:24:55 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.240791776s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X..XX............................................] duration: 2.06887798s FAIL (2025-08-31 13:24:54 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.068910462s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [................X.X.........................................] duration: 744.976926ms target: https://us1.storj.io/login FAIL (2025-08-31 13:24:54 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 745.001023ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.128583149s FAIL (2025-08-31 13:24:47 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.128602666s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.................X.X........................................] duration: 753.929527ms target: https://us1.storj.io/login FAIL (2025-08-31 13:24:36 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 753.946368ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X..XX...........................................] duration: 2.216084941s FAIL (2025-08-31 13:24:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.216108966s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration: 757.643693ms target: https://us1.storj.io/login FAIL (2025-08-31 13:24:03 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 757.667609ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 6.034013211s FAIL (2025-08-31 13:23:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.034036636s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X..XX..........................................] duration: 2.108746317s FAIL (2025-08-31 13:23:44 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.108771014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 6.253952491s FAIL (2025-08-31 13:23:44 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.253982108s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...................X.X......................................] duration: 751.706764ms target: https://us1.storj.io/login FAIL (2025-08-31 13:23:40 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 751.733256ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.124437339s FAIL (2025-08-31 13:23:35 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.124465472s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 6.160274011s FAIL (2025-08-31 13:23:12 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.160294892s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration: 816.621923ms target: https://us1.storj.io/login FAIL (2025-08-31 13:23:11 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 816.654234ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X..XX.........................................] duration: 2.369103149s FAIL (2025-08-31 13:23:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.369138355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 6.031945103s FAIL (2025-08-31 13:22:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.031979569s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.265858817s FAIL (2025-08-31 13:22:43 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.265880229s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 6.063508277s FAIL (2025-08-31 13:22:21 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.063536741s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X..XX........................................] duration: 2.244015863s FAIL (2025-08-31 13:22:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.244158174s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.X....................................] duration: 761.739058ms target: https://us1.storj.io/login FAIL (2025-08-31 13:22:02 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 761.757524ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.365434802s FAIL (2025-08-31 13:22:01 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.365460972s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 6.459828466s FAIL (2025-08-31 13:21:59 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.459854495s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 5.924639959s FAIL (2025-08-31 13:21:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.924662803s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 6.073263849s FAIL (2025-08-31 13:21:20 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.073288395s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X..XX.......................................] duration: 2.131416592s FAIL (2025-08-31 13:21:05 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.13144248s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.987904671s FAIL (2025-08-31 13:21:05 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.987931393s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [......................X.X...................................] duration: 675.313715ms target: https://us1.storj.io/login FAIL (2025-08-31 13:21:04 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 675.336467ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 6.066525278s FAIL (2025-08-31 13:21:01 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.066555906s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X..XX......................................] duration: 1.957004485s FAIL (2025-08-31 13:20:43 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 1.957028953s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 6.143965204s FAIL (2025-08-31 13:20:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.143991212s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.X..................................] duration: 699.826127ms target: https://us1.storj.io/login FAIL (2025-08-31 13:20:29 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 699.846386ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X..XX.....................................] duration: 2.264843794s FAIL (2025-08-31 13:20:27 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.264871508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 6.321188889s FAIL (2025-08-31 13:20:27 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.32128392s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.020573611s FAIL (2025-08-31 13:20:24 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.020626001s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.08610393s FAIL (2025-08-31 13:20:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.086130921s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 5.982548424s FAIL (2025-08-31 13:20:11 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.982588431s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.206782939s FAIL (2025-08-31 13:20:00 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.206809167s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [........................X.X.................................] duration: 766.860341ms target: https://us1.storj.io/login FAIL (2025-08-31 13:19:52 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 766.894908ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 6.401208327s FAIL (2025-08-31 13:19:40 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.401256629s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 6.022316646s FAIL (2025-08-31 13:19:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.0223444s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................X..XX....................................] duration: 2.176425373s FAIL (2025-08-31 13:19:18 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.176444599s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 6.067937342s FAIL (2025-08-31 13:19:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54:20.466112488 (46h46m17.146308057s) [upload=41121]: Starting upload Aug 31 12:54:20.466181300 (46h46m17.146376869s) [upload=41121]: Done starting upload Aug 31 12:54:20.466293645 (46h46m17.146489214s) [upload=41121]: Uploading segments... Aug 31 12:54:20.466415478 (46h46m17.146611047s) [upload=41121]: Got next segment. Inline: false Aug 31 12:54:20.466452830 (46h46m17.146648399s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:54:20.466457549 (46h46m17.146653118s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 31 12:54:21.294424434 (46h46m17.974620003s) [seg_pos={0 0}, upload=41121]: Begin upload segment... Aug 31 12:54:21.294491232 (46h46m17.974686801s) [seg_pos={0 0}, upload=41121]: Done begin upload segment. Aug 31 12:54:21.294528533 (46h46m17.974724102s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294553080 (46h46m17.974748649s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294599369 (46h46m17.974794938s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.294949769 (46h46m17.975145338s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295004123 (46h46m17.975199692s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295241668 (46h46m17.975437237s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295363952 (46h46m17.975559521s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295598250 (46h46m17.975793809s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.295637255 (46h46m17.975832824s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296028964 (46h46m17.976224533s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.296054323 (46h46m17.976249902s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.297406550 (46h46m17.977602119s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298472500 (46h46m17.978668069s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.298936327 (46h46m17.979131896s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.299472263 (46h46m17.979667832s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300081980 (46h46m17.980277549s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.300620470 (46h46m17.980816039s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:21.301163599 (46h46m17.981359158s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Uploading piece... Aug 31 12:54:22.445988099 (46h46m19.126183668s) [upload=41121]: Next returned nil segment Aug 31 12:54:22.446001505 (46h46m19.126197065s) [upload=41121]: Waiting for error group managing segments... Aug 31 12:54:22.736812151 (46h46m19.41700772s) [noise=true , address=79.127.205.245:4244 , piece_id=6HXI6PPILQG3C7X4KYEA3ZAX4Z44LS4KW7GQEVQK22ZX722BGTGQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:22.739141308 (46h46m19.419336867s) [noise=true , address=79.127.205.246:4263 , piece_id=5DEIYDRBOZPDLTQDBVC6F47U7MLVZZ627WIWJUHYW4MCXDKZE3YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.224539823 (46h46m19.904735392s) [noise=true , address=45.91.116.12:4230 , piece_id=KD3LU6PNFIN6GMDZHJMNJY4HBPIZETMG6TXMPPZLZL3WSRZ3RN4Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.271009900 (46h46m19.951205469s) [noise=true , address=77.90.31.16:4752 , piece_id=AIHJKVIZOHP4K2RK3UDHRI5KNOZLQ6ZEMRCUG52XO3V67QTR3NKA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.276455820 (46h46m19.956651389s) [noise=true , address=104.36.82.21:4475 , piece_id=FRZXAJKN56BRN5SF6AUIZI4YVZ523TA6AHFDZBQTBNNGCOIK56UQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.296846557 (46h46m19.977042126s) [noise=true , address=104.36.82.47:4543 , piece_id=7SULW2ORT4MBDCNR2ZB4YSZY3DSK5J73HNZBEOWFSEKDI4FQEGMQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.317956980 (46h46m19.998152549s) [noise=true , address=77.90.31.18:4814 , piece_id=E6BX56GD5TRV3DM7XND37WKEY4M5RBSLEQEU6ALJGG5LNZEMYG6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.321357937 (46h46m20.001553506s) [noise=true , address=104.36.82.13:4219 , piece_id=4ZNQYJMRANHS5JFBXFEJPI2YD323ZTAT5KSW6U2BYQ3PLLTR63DA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.323031369 (46h46m20.003226938s) [noise=true , address=104.36.82.5:4842 , piece_id=TCQZJFUSWMCFV2FNIRGSCDKDYS3ITXFATVCAGIN4PRPUCAZABGJQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.326882398 (46h46m20.007077967s) [noise=true , address=104.36.82.15:4273 , piece_id=FECGOGMOAPU2NP7M5BYE6CHVZKZBXIVCSXCEA5H6S2WZZO2IUM3Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.334483402 (46h46m20.014678971s) [noise=true , address=104.36.82.6:4888 , piece_id=REU6R7TCSGR4UZKVSQF2Z5KT3PUI4EGI4ZMN6EDZGGRAPTJVH74Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.349026589 (46h46m20.029222158s) [noise=true , address=77.90.31.17:4795 , piece_id=UI6OHARYSVJ6J2I3FHKMPHHR2UPYE7CGPWVQJAEZ5MDDIHCDHROA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.353189805 (46h46m20.033385374s) [noise=true , address=104.36.82.48:4579 , piece_id=EZ4AJQ2JYWNSPBJC5UTPYZEGZZNTO2MEXK2YEMOB3SLIIGFLKVBA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357943061 (46h46m20.03813863s) [noise=true , address=104.36.82.40:4962 , piece_id=XBLGHDBFFEYKVDKJQLHIRRKPMLSM57MVGJ6HQNMTV4VCOXTEWB6Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: <nil> Aug 31 12:54:23.357966986 (46h46m20.038162555s) [seg_pos={0 0}, upload=41121]: Segment reached optimal threshold of 14 pieces. Aug 31 12:54:23.358105080 (46h46m20.038300639s) [noise=true , address=104.36.82.14:4227 , piece_id=5LNQ3CZJMD4B6VWJNANXVPOFTG6ZCAALVRW2QGZU5F3WBRQKDN5Q, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358182047 (46h46m20.038377617s) [noise=true , address=104.36.82.44:4875 , piece_id=IU2FPY3ZWINTTVPZVNUFY6APOKHR7DGWQUKF4AGXQSLUSCZ6ULEA, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358209420 (46h46m20.038404989s) [noise=true , address=104.36.82.46:4497 , piece_id=CERGPCKFKAZIXDCAAWEU2KS35KVNJX3PYR2DFYHULULR5D4CS2VQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358269535 (46h46m20.038465104s) [noise=true , address=104.36.82.60:4339 , piece_id=MFUIHLXIZMZK5NSUVQZLDWPFPF5BJV4UY5ESOUGWR2FFJ6LV36YQ, seg_pos={0 0}, upload=41121]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R): protocol: expected piece hash; context canceled; EOF Aug 31 12:54:23.358293110 (46h46m20.038488679s) [seg_pos={0 0}, upload=41121]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 31 12:54:23.358370789 (46h46m20.038566358s) [upload=41121]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 31 12:54:23.358376369 (46h46m20.038571938s) [upload=41121]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:56:26.170691241 (46h48m22.85088682s) [upload=41121]: Done uploading segments... Aug 31 12:56:26.170715998 (46h48m22.850911567s) [upload=41121]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.067974763s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.11057343s FAIL (2025-08-31 13:19:00 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.110598989s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................X..XX...................................] duration: 2.238274164s FAIL (2025-08-31 13:18:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 31 12:55:03.441619160 (46h47m0.121814729s) [upload=41130]: Starting upload Aug 31 12:55:03.441712168 (46h47m0.121907767s) [upload=41130]: Done starting upload Aug 31 12:55:03.441856173 (46h47m0.122051772s) [upload=41130]: Uploading segments... Aug 31 12:55:03.441960683 (46h47m0.122156252s) [upload=41130]: Got next segment. Inline: true Aug 31 12:55:03.441966745 (46h47m0.122162324s) [upload=41130]: Waiting for error group managing segments... Aug 31 12:55:03.442016209 (46h47m0.122211779s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 31 12:55:03.442021519 (46h47m0.122217088s) [upload=41130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 31 12:55:03.442024626 (46h47m0.122220195s) [upload=41130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 31 12:55:03.946537997 (46h47m0.626733566s) [upload=41130]: Done uploading segments... Aug 31 12:55:03.946558877 (46h47m0.626754446s) [upload=41130]: Upload finished. err: <nil> Duration:* 2.238299945s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.X................................] duration: 735.194241ms target: https://us1.storj.io/login FAIL (2025-08-31 13:18:54 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 735.222404ms core satellite us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)