Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [........................XXXXXXXXXXXXXXXXXXXXXXXXXX...X..X...] duration: 6.259077772s FAIL (2025-08-07 21:49:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.259120535s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [..........................X....X.X...X......................] duration: 738.591167ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:49:26 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 738.627808ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [..............X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.X...] duration: 9.433283211s FAIL (2025-08-07 21:49:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection Aug 7 21:44:51.028127350 (12h4m50.617472327s) [seg_pos={0 0}, upload=12555]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:44:51.028226048 (12h4m50.617571035s) [upload=12555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:51.028237751 (12h4m50.617582738s) [upload=12555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:51.322864709 (12h4m50.912209697s) [upload=12555]: Done uploading segments... Aug 7 21:44:51.322882523 (12h4m50.9122275s) [upload=12555]: Upload finished. err: <nil> Aug 7 21:45:58.815791269 (12h5m58.405136256s) [upload=12571]: Starting upload Aug 7 21:45:58.815846996 (12h5m58.405191983s) [upload=12571]: Done starting upload Aug 7 21:45:58.815953540 (12h5m58.405298527s) [upload=12571]: Uploading segments... Aug 7 21:45:58.816065333 (12h5m58.405410321s) [upload=12571]: Got next segment. Inline: false Aug 7 21:45:58.816100661 (12h5m58.405445638s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.816106212 (12h5m58.405451199s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:59.064845605 (12h5m58.654190592s) [seg_pos={0 0}, upload=12571]: Begin upload segment... Aug 7 21:45:59.064885521 (12h5m58.654230508s) [seg_pos={0 0}, upload=12571]: Done begin upload segment. Aug 7 21:45:59.064910399 (12h5m58.654255376s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.064952560 (12h5m58.654297547s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065245920 (12h5m58.654590907s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065360070 (12h5m58.654705057s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065517019 (12h5m58.654861996s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065615889 (12h5m58.654960876s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065778009 (12h5m58.655122986s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065884293 (12h5m58.65522928s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066024761 (12h5m58.655369748s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066243109 (12h5m58.655588096s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066452850 (12h5m58.655797837s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066669685 (12h5m58.656014672s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066955041 (12h5m58.656300028s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067149393 (12h5m58.65649438s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067334066 (12h5m58.656679053s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067527215 (12h5m58.656872202s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:46:00.855806043 (12h6m0.44515103s) [upload=12571]: Next returned nil segment Aug 7 21:46:00.855821061 (12h6m0.445166048s) [upload=12571]: Waiting for error group managing segments... Aug 7 21:46:01.180369497 (12h6m0.769714484s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.200492297 (12h6m0.789837284s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.210262691 (12h6m0.799607678s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.220625478 (12h6m0.809970465s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.304447281 (12h6m0.893792308s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.374462398 (12h6m0.963807385s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.378351098 (12h6m0.967696085s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.395257154 (12h6m0.984602141s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.404312138 (12h6m0.993657125s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.409522908 (12h6m0.998867885s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.416315134 (12h6m1.005660121s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.421432725 (12h6m1.010777712s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439709112 (12h6m1.029054099s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439735472 (12h6m1.029080459s) [seg_pos={0 0}, upload=12571]: Segment reached optimal threshold of 13 pieces. Aug 7 21:46:01.440013234 (12h6m1.029358221s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440020518 (12h6m1.029365505s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: write tcp 188.166.225.170:54068->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:46:01.440109398 (12h6m1.029454385s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440122312 (12h6m1.029467299s) [seg_pos={0 0}, upload=12571]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:46:01.440193058 (12h6m1.029538045s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:46:01.440197016 (12h6m1.029542003s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:14.557737130 (12h8m14.147082107s) [upload=12571]: Done uploading segments... Aug 7 21:48:14.557758951 (12h8m14.147103938s) [upload=12571]: Upload finished. err: metaclient: context deadline exceeded Duration: 9.43330928s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [...........XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X............] duration: 2.267090939s FAIL (2025-08-07 21:49:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.267121317s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 26 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXXXXXXXXXXXXXXX.....X..] duration: 7.454341256s FAIL (2025-08-07 21:49:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== t segment. Inline: false Aug 7 21:44:25.061589945 (12h4m24.650934922s) [upload=12552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:25.061594724 (12h4m24.650939711s) [upload=12552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:44:50.602885060 (12h4m50.192230047s) [seg_pos={0 0}, upload=12552]: Begin upload segment... Aug 7 21:44:50.602935446 (12h4m50.192280423s) [seg_pos={0 0}, upload=12552]: Done begin upload segment. Aug 7 21:44:50.603112565 (12h4m50.192457552s) [noise=true , address=79.127.205.233:4624 , piece_id=WEAYHGF7VE6W5ML3QAE3AITMEJGQFH4JQZ7RQKR7O4NKTPR6RZRA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.603695610 (12h4m50.193040597s) [noise=true , address=104.36.82.22:4529 , piece_id=ZTTBM5H6MHVMQ3OFVTVISN6WJ7WB3XBWI7YZKUXCFNQ6NO3D4CGA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.604300768 (12h4m50.193645755s) [noise=true , address=45.91.116.6:4002 , piece_id=AE5EL7NWD6FEQ7NXX2JRHB4MVZQDXRGQU3EER2X7COISFAX6P3XQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.604588519 (12h4m50.193933496s) [noise=true , address=104.36.82.56:4249 , piece_id=RD4B7IDIJV25L7EJNSYCZGW5QKJEM435E2N3UFUD44Q4S3DCBOXQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605153860 (12h4m50.194498837s) [noise=true , address=77.90.31.11:4562 , piece_id=XC6F2WLSXKQB7E7MBRNZBJKUSUFL6XNA5TVX5M3ZJI3ZZMK7UBLA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605455377 (12h4m50.194800364s) [noise=true , address=104.36.82.23:4466 , piece_id=5J5X2LSP6GCX2PNRIDZYFJPISWSX3YSRID5M2WU6Y5Z6LWRHLC4Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605743810 (12h4m50.195088797s) [noise=true , address=104.36.82.62:4420 , piece_id=UOO36PYVE5WFRRZLGYLJ6I6SV4EO7W2ZH2TIXJ2OIZZMY3LTHR2A, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.606547457 (12h4m50.195892444s) [noise=true , address=104.36.82.5:4842 , piece_id=E5DSJXN63RFX7EQ4ZTBMUESFW77EB3HQNA5KCQGRVRAA5XO2GN3Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.606982099 (12h4m50.196327086s) [noise=true , address=104.36.82.59:4301 , piece_id=VVR5CQWCPNPZUFEFCPB6ZL2KASWS5XO65FVDI474RKRZSJFDUQTA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.607295849 (12h4m50.196640836s) [noise=true , address=45.91.116.14:4198 , piece_id=2HRU3JZDQSNGD5NJZDLZSRG2WIO2Y2QBSOAY6KUEGUKDBWM53RSA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.607695593 (12h4m50.19704058s) [noise=true , address=104.36.82.6:4881 , piece_id=VUC33D7TLVFICDLA3334DEKBJE5E5P4HQTJU3VDQNMVYWVESSJWA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.608364103 (12h4m50.19770909s) [noise=true , address=104.36.82.2:4757 , piece_id=H6KNYP5HASPWQIHADTMEZDC5AE5HITCAY73XGNWVEXJOPDEZ3SHA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.608764288 (12h4m50.198109265s) [noise=true , address=104.36.82.46:4493 , piece_id=PS4S4OZ6HSITEJFPJDIBWQ5RGWTDRKMX3NCEWXCLC2PSP5Q2KPZQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609065284 (12h4m50.198410271s) [noise=true , address=104.36.82.47:4518 , piece_id=KJ3PJCT3EDH3WZBT7AWWKI6ABGVVAMKZTVJY7Q76FULC72TUWA6Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609312777 (12h4m50.198657755s) [noise=true , address=104.36.82.60:4349 , piece_id=GTTF32YMJWBY2GZEFZPIBZXZWNI7MBLUYCTQ3SRFG735PLWXTXBQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609589888 (12h4m50.198934875s) [noise=true , address=104.36.82.52:4116 , piece_id=7QA2Q5ACN43TQUOKWA34EZFSSQBE3OWNJTYODHHQ5GCVOA4QZOJA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.610295969 (12h4m50.199641056s) [noise=true , address=104.36.82.37:4678 , piece_id=VJO5JO3SJEXBQZC6PJ34DRKWEQKOPI33ZF4IHNNOOTSD7IBRR4PQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.610811305 (12h4m50.200156302s) [noise=true , address=64.49.13.169:4714 , piece_id=A6O3H7QLE6KQTHWUKOEROVHPEHVLG22KNF2KCV3HK2KCLXP7SYVA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:51.715421583 (12h4m51.30476657s) [upload=12552]: Next returned nil segment Aug 7 21:44:51.715438125 (12h4m51.304783112s) [upload=12552]: Waiting for error group managing segments... Aug 7 21:44:51.998131281 (12h4m51.587476268s) [noise=true , address=64.49.13.169:4714 , piece_id=A6O3H7QLE6KQTHWUKOEROVHPEHVLG22KNF2KCV3HK2KCLXP7SYVA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:51.999236335 (12h4m51.588581312s) [noise=true , address=79.127.205.233:4624 , piece_id=WEAYHGF7VE6W5ML3QAE3AITMEJGQFH4JQZ7RQKR7O4NKTPR6RZRA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.638299991 (12h4m52.227644978s) [noise=true , address=45.91.116.6:4002 , piece_id=AE5EL7NWD6FEQ7NXX2JRHB4MVZQDXRGQU3EER2X7COISFAX6P3XQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.639927565 (12h4m52.229272542s) [noise=true , address=104.36.82.6:4881 , piece_id=VUC33D7TLVFICDLA3334DEKBJE5E5P4HQTJU3VDQNMVYWVESSJWA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.643220585 (12h4m52.232565572s) [noise=true , address=104.36.82.2:4757 , piece_id=H6KNYP5HASPWQIHADTMEZDC5AE5HITCAY73XGNWVEXJOPDEZ3SHA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.711872926 (12h4m52.301217913s) [noise=true , address=104.36.82.56:4249 , piece_id=RD4B7IDIJV25L7EJNSYCZGW5QKJEM435E2N3UFUD44Q4S3DCBOXQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.763601896 (12h4m52.352946883s) [noise=true , address=104.36.82.22:4529 , piece_id=ZTTBM5H6MHVMQ3OFVTVISN6WJ7WB3XBWI7YZKUXCFNQ6NO3D4CGA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.768238797 (12h4m52.357583785s) [noise=true , address=104.36.82.60:4349 , piece_id=GTTF32YMJWBY2GZEFZPIBZXZWNI7MBLUYCTQ3SRFG735PLWXTXBQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.798543408 (12h4m52.387888395s) [noise=true , address=45.91.116.14:4198 , piece_id=2HRU3JZDQSNGD5NJZDLZSRG2WIO2Y2QBSOAY6KUEGUKDBWM53RSA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.861192704 (12h4m52.450537691s) [noise=true , address=77.90.31.11:4562 , piece_id=XC6F2WLSXKQB7E7MBRNZBJKUSUFL6XNA5TVX5M3ZJI3ZZMK7UBLA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:53.070218164 (12h4m52.659563151s) [noise=true , address=104.36.82.59:4301 , piece_id=VVR5CQWCPNPZUFEFCPB6ZL2KASWS5XO65FVDI474RKRZSJFDUQTA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:53.558930346 (12h4m53.148275333s) [noise=true , address=104.36.82.46:4493 , piece_id=PS4S4OZ6HSITEJFPJDIBWQ5RGWTDRKMX3NCEWXCLC2PSP5Q2KPZQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.132342777 (12h4m54.721687765s) [noise=true , address=104.36.82.37:4678 , piece_id=VJO5JO3SJEXBQZC6PJ34DRKWEQKOPI33ZF4IHNNOOTSD7IBRR4PQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.265236558 (12h4m54.854581545s) [noise=true , address=104.36.82.52:4116 , piece_id=7QA2Q5ACN43TQUOKWA34EZFSSQBE3OWNJTYODHHQ5GCVOA4QZOJA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.265254161 (12h4m54.854599138s) [seg_pos={0 0}, upload=12552]: Segment reached optimal threshold of 14 pieces. Aug 7 21:44:55.265482308 (12h4m54.854827295s) [noise=true , address=104.36.82.5:4842 , piece_id=E5DSJXN63RFX7EQ4ZTBMUESFW77EB3HQNA5KCQGRVRAA5XO2GN3Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn): protocol: expected piece hash; write tcp 188.166.225.170:57020->104.36.82.5:4842: use of closed network connection; write tcp 188.166.225.170:57020->104.36.82.5:4842: use of closed network connection; EOF Aug 7 21:44:55.265551430 (12h4m54.854896417s) [noise=true , address=104.36.82.47:4518 , piece_id=KJ3PJCT3EDH3WZBT7AWWKI6ABGVVAMKZTVJY7Q76FULC72TUWA6Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17TZ9JLdCivFF46guhSTW6hcoUmTz5uo5wvLkDqC4BGmpm9eLb): protocol: expected piece hash; context canceled; EOF Aug 7 21:44:55.265766391 (12h4m54.855111378s) [noise=true , address=104.36.82.23:4466 , piece_id=5J5X2LSP6GCX2PNRIDZYFJPISWSX3YSRID5M2WU6Y5Z6LWRHLC4Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: write tcp 188.166.225.170:49724->104.36.82.23:4466: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:44:55.265824172 (12h4m54.855169159s) [noise=true , address=104.36.82.62:4420 , piece_id=UOO36PYVE5WFRRZLGYLJ6I6SV4EO7W2ZH2TIXJ2OIZZMY3LTHR2A, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SWVQwL5PxDLaSXADwTrAYS2hVcNBuYAA6ReWuRM3UZa57QyYi): protocol: read tcp 188.166.225.170:33092->104.36.82.62:4420: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:44:55.265833430 (12h4m54.855178417s) [seg_pos={0 0}, upload=12552]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:44:55.265888796 (12h4m54.855233783s) [upload=12552]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:55.265892303 (12h4m54.85523728s) [upload=12552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:46.088275738 (12h8m45.677620725s) [upload=12552]: Done uploading segments... Aug 7 21:48:46.088301307 (12h8m45.677646284s) [upload=12552]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.454389066s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X.......] duration: 1.954591825s FAIL (2025-08-07 21:49:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.954616011s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [.............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.......X..] duration: 6.637900799s FAIL (2025-08-07 21:49:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.637928333s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [...................................XX.XX..X.XX..............] duration: 290.010296ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:49:14 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 290.03334ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 32 out of 60 is failed (threshold=3) [.....................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..X...] duration: 2.00421561s FAIL (2025-08-07 21:49:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 6664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:11.998653609 (12h5m11.587998596s) [upload=12563]: Starting upload Aug 7 21:45:11.998732921 (12h5m11.588077908s) [upload=12563]: Done starting upload Aug 7 21:45:11.998811802 (12h5m11.588156789s) [upload=12563]: Uploading segments... Aug 7 21:45:11.998922443 (12h5m11.58826743s) [upload=12563]: Got next segment. Inline: true Aug 7 21:45:11.998926992 (12h5m11.588271979s) [upload=12563]: Waiting for error group managing segments... Aug 7 21:45:11.998979913 (12h5m11.5883249s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:11.998987267 (12h5m11.588332254s) [upload=12563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:11.998990603 (12h5m11.58833559s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:14.137150454 (12h5m13.726495441s) [upload=12563]: Done uploading segments... Aug 7 21:45:14.137168529 (12h5m13.726513516s) [upload=12563]: Upload finished. err: <nil> Aug 7 21:45:53.851988781 (12h5m53.441333768s) [upload=12569]: Starting upload Aug 7 21:45:53.852043686 (12h5m53.441388673s) [upload=12569]: Done starting upload Aug 7 21:45:53.852088772 (12h5m53.441433759s) [upload=12569]: Uploading segments... Aug 7 21:45:53.852141613 (12h5m53.4414866s) [upload=12569]: Got next segment. Inline: true Aug 7 21:45:53.852146192 (12h5m53.441491169s) [upload=12569]: Waiting for error group managing segments... Aug 7 21:45:53.852182612 (12h5m53.441527599s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:53.852186920 (12h5m53.441531907s) [upload=12569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:53.852189195 (12h5m53.441534182s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:54.127817304 (12h5m53.717162291s) [upload=12569]: Done uploading segments... Aug 7 21:45:54.127834497 (12h5m53.717179484s) [upload=12569]: Upload finished. err: <nil> Aug 7 21:46:54.936546775 (12h6m54.525891762s) [upload=12585]: Starting upload Aug 7 21:46:54.936595597 (12h6m54.525940585s) [upload=12585]: Done starting upload Aug 7 21:46:54.936645253 (12h6m54.52599024s) [upload=12585]: Uploading segments... Aug 7 21:46:54.936675961 (12h6m54.526020948s) [upload=12585]: Got next segment. Inline: true Aug 7 21:46:54.936680381 (12h6m54.526025358s) [upload=12585]: Waiting for error group managing segments... Aug 7 21:46:54.936716089 (12h6m54.526061076s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:54.936720797 (12h6m54.526065774s) [upload=12585]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:54.936723172 (12h6m54.526068159s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:54.288569811 (12h7m53.877914798s) [upload=12585]: Done uploading segments... Aug 7 21:47:54.288620218 (12h7m53.877965205s) [upload=12585]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.004250858s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 33 out of 60 is failed (threshold=3) [..............X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X........] duration: 1.939381285s FAIL (2025-08-07 21:49:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.939407766s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X.........X......X..X.............X..X............] duration: 7.509444723s FAIL (2025-08-07 21:49:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== ad=12478]: Done starting upload Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments... Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment... Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment. Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments... Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces. Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments... Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil> Duration: 7.509474911s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...............X..........X.........] duration: 1.94118174s FAIL (2025-08-07 21:48:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 735125827s) [upload=12376]: Done uploading segments... Aug 7 21:34:02.145826196 (11h54m1.735171183s) [upload=12376]: Upload finished. err: <nil> Aug 7 21:34:47.873603965 (11h54m47.462948952s) [upload=12391]: Starting upload Aug 7 21:34:47.873650053 (11h54m47.46299503s) [upload=12391]: Done starting upload Aug 7 21:34:47.873700940 (11h54m47.463045928s) [upload=12391]: Uploading segments... Aug 7 21:34:47.873733743 (11h54m47.46307872s) [upload=12391]: Got next segment. Inline: true Aug 7 21:34:47.873739335 (11h54m47.463084322s) [upload=12391]: Waiting for error group managing segments... Aug 7 21:34:47.873774331 (11h54m47.463119318s) [upload=12391]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:47.873777988 (11h54m47.463122975s) [upload=12391]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:47.873780623 (11h54m47.46312561s) [upload=12391]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.144972113 (11h54m47.7343171s) [upload=12391]: Done uploading segments... Aug 7 21:34:48.144991410 (11h54m47.734336397s) [upload=12391]: Upload finished. err: <nil> Aug 7 21:35:20.363321680 (11h55m19.952666667s) [upload=12399]: Starting upload Aug 7 21:35:20.363369802 (11h55m19.952714789s) [upload=12399]: Done starting upload Aug 7 21:35:20.363434526 (11h55m19.952779513s) [upload=12399]: Uploading segments... Aug 7 21:35:20.363500943 (11h55m19.95284597s) [upload=12399]: Got next segment. Inline: true Aug 7 21:35:20.363506363 (11h55m19.95285135s) [upload=12399]: Waiting for error group managing segments... Aug 7 21:35:20.363545789 (11h55m19.952890776s) [upload=12399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:20.363550498 (11h55m19.952895485s) [upload=12399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:20.363552953 (11h55m19.95289793s) [upload=12399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:20.648472420 (11h55m20.237817407s) [upload=12399]: Done uploading segments... Aug 7 21:35:20.648492398 (11h55m20.237837386s) [upload=12399]: Upload finished. err: <nil> Aug 7 21:35:35.916328511 (11h55m35.505673488s) [upload=12404]: Starting upload Aug 7 21:35:35.916377014 (11h55m35.505722001s) [upload=12404]: Done starting upload Aug 7 21:35:35.916425818 (11h55m35.505770805s) [upload=12404]: Uploading segments... Aug 7 21:35:35.916457979 (11h55m35.505802956s) [upload=12404]: Got next segment. Inline: true Aug 7 21:35:35.916461466 (11h55m35.505806443s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:35:35.916496753 (11h55m35.50584173s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:35.916501372 (11h55m35.505846359s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:35.916503666 (11h55m35.505848653s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:36.531148061 (11h55m36.120493049s) [upload=12404]: Done uploading segments... Aug 7 21:35:36.531167528 (11h55m36.120512505s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:36:48.991080864 (11h56m48.580425851s) [upload=12421]: Starting upload Aug 7 21:36:48.991131251 (11h56m48.580476238s) [upload=12421]: Done starting upload Aug 7 21:36:48.991177239 (11h56m48.580522226s) [upload=12421]: Uploading segments... Aug 7 21:36:48.991205413 (11h56m48.58055039s) [upload=12421]: Got next segment. Inline: true Aug 7 21:36:48.991209060 (11h56m48.580554037s) [upload=12421]: Waiting for error group managing segments... Aug 7 21:36:48.991242914 (11h56m48.580587901s) [upload=12421]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:48.991246341 (11h56m48.580591328s) [upload=12421]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:48.991248505 (11h56m48.580593492s) [upload=12421]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:49.270270137 (11h56m48.859615134s) [upload=12421]: Done uploading segments... Aug 7 21:36:49.270288603 (11h56m48.85963359s) [upload=12421]: Upload finished. err: <nil> Aug 7 21:37:34.116423494 (11h57m33.705768481s) [upload=12435]: Starting upload Aug 7 21:37:34.116478008 (11h57m33.705822996s) [upload=12435]: Done starting upload Aug 7 21:37:34.116546550 (11h57m33.705891537s) [upload=12435]: Uploading segments... Aug 7 21:37:34.116585414 (11h57m33.705930401s) [upload=12435]: Got next segment. Inline: true Aug 7 21:37:34.116589281 (11h57m33.705934268s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:37:34.116631473 (11h57m33.70597646s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:34.116635470 (11h57m33.705980457s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:34.116638065 (11h57m33.705983042s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:34.430160454 (11h57m34.019505441s) [upload=12435]: Done uploading segments... Aug 7 21:37:34.430179780 (11h57m34.019524767s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:38:07.899363525 (11h58m7.488708512s) [upload=12444]: Starting upload Aug 7 21:38:07.899432257 (11h58m7.488777254s) [upload=12444]: Done starting upload Aug 7 21:38:07.899478635 (11h58m7.488823622s) [upload=12444]: Uploading segments... Aug 7 21:38:07.899513011 (11h58m7.488857998s) [upload=12444]: Got next segment. Inline: true Aug 7 21:38:07.899516348 (11h58m7.488861325s) [upload=12444]: Waiting for error group managing segments... Aug 7 21:38:07.899544021 (11h58m7.488888998s) [upload=12444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:07.899547738 (11h58m7.488892725s) [upload=12444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:07.899551905 (11h58m7.488896892s) [upload=12444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:08.175487142 (11h58m7.764832129s) [upload=12444]: Done uploading segments... Aug 7 21:38:08.175506650 (11h58m7.764851637s) [upload=12444]: Upload finished. err: <nil> Aug 7 21:39:12.808926452 (11h59m12.398271439s) [upload=12459]: Starting upload Aug 7 21:39:12.808989873 (11h59m12.39833486s) [upload=12459]: Done starting upload Aug 7 21:39:12.809038286 (11h59m12.398383273s) [upload=12459]: Uploading segments... Aug 7 21:39:12.809068093 (11h59m12.39841308s) [upload=12459]: Got next segment. Inline: true Aug 7 21:39:12.809072080 (11h59m12.398417068s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:39:12.809118720 (11h59m12.398463757s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:12.809125252 (11h59m12.398470239s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:12.809127897 (11h59m12.398472884s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:13.234618442 (11h59m12.823963429s) [upload=12459]: Done uploading segments... Aug 7 21:39:13.234636597 (11h59m12.823981584s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:39:55.593720326 (11h59m55.183065313s) [upload=12472]: Starting upload Aug 7 21:39:55.593767186 (11h59m55.183112163s) [upload=12472]: Done starting upload Aug 7 21:39:55.593842129 (11h59m55.183187117s) [upload=12472]: Uploading segments... Aug 7 21:39:55.593879050 (11h59m55.183224037s) [upload=12472]: Got next segment. Inline: true Aug 7 21:39:55.593882546 (11h59m55.183227533s) [upload=12472]: Waiting for error group managing segments... Aug 7 21:39:55.593915990 (11h59m55.183260978s) [upload=12472]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:55.593919928 (11h59m55.183264916s) [upload=12472]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:55.593922323 (11h59m55.18326731s) [upload=12472]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:55.991096020 (11h59m55.580441007s) [upload=12472]: Done uploading segments... Aug 7 21:39:55.991111470 (11h59m55.580456457s) [upload=12472]: Upload finished. err: <nil> Aug 7 21:40:19.545259354 (12h0m19.134604341s) [upload=12480]: Starting upload Aug 7 21:40:19.545304791 (12h0m19.134649778s) [upload=12480]: Done starting upload Aug 7 21:40:19.545361891 (12h0m19.134706878s) [upload=12480]: Uploading segments... Aug 7 21:40:19.545393461 (12h0m19.134738448s) [upload=12480]: Got next segment. Inline: true Aug 7 21:40:19.545397098 (12h0m19.134742085s) [upload=12480]: Waiting for error group managing segments... Aug 7 21:40:19.545429900 (12h0m19.134774887s) [upload=12480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:19.545433667 (12h0m19.134778655s) [upload=12480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:19.545435852 (12h0m19.134780839s) [upload=12480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:19.891464235 (12h0m19.480809212s) [upload=12480]: Done uploading segments... Aug 7 21:40:19.891480988 (12h0m19.480825965s) [upload=12480]: Upload finished. err: <nil> Aug 7 21:41:23.105250182 (12h1m22.694595169s) [upload=12497]: Starting upload Aug 7 21:41:23.105299878 (12h1m22.694644865s) [upload=12497]: Done starting upload Aug 7 21:41:23.105354182 (12h1m22.694699169s) [upload=12497]: Uploading segments... Aug 7 21:41:23.105392174 (12h1m22.694737161s) [upload=12497]: Got next segment. Inline: true Aug 7 21:41:23.105395991 (12h1m22.694740978s) [upload=12497]: Waiting for error group managing segments... Aug 7 21:41:23.105430738 (12h1m22.694775715s) [upload=12497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:23.105434485 (12h1m22.694779472s) [upload=12497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:23.105438433 (12h1m22.69478342s) [upload=12497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:23.395695219 (12h1m22.985040206s) [upload=12497]: Done uploading segments... Aug 7 21:41:23.395713514 (12h1m22.985058501s) [upload=12497]: Upload finished. err: <nil> Duration: 1.941203782s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...........] duration: 2.531687181s FAIL (2025-08-07 21:48:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.531710005s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 26 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXXXXXXXXXXXXXXX.....X.] duration: 6.712611674s FAIL (2025-08-07 21:48:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== t segment. Inline: false Aug 7 21:44:25.061589945 (12h4m24.650934922s) [upload=12552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:25.061594724 (12h4m24.650939711s) [upload=12552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:44:50.602885060 (12h4m50.192230047s) [seg_pos={0 0}, upload=12552]: Begin upload segment... Aug 7 21:44:50.602935446 (12h4m50.192280423s) [seg_pos={0 0}, upload=12552]: Done begin upload segment. Aug 7 21:44:50.603112565 (12h4m50.192457552s) [noise=true , address=79.127.205.233:4624 , piece_id=WEAYHGF7VE6W5ML3QAE3AITMEJGQFH4JQZ7RQKR7O4NKTPR6RZRA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.603695610 (12h4m50.193040597s) [noise=true , address=104.36.82.22:4529 , piece_id=ZTTBM5H6MHVMQ3OFVTVISN6WJ7WB3XBWI7YZKUXCFNQ6NO3D4CGA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.604300768 (12h4m50.193645755s) [noise=true , address=45.91.116.6:4002 , piece_id=AE5EL7NWD6FEQ7NXX2JRHB4MVZQDXRGQU3EER2X7COISFAX6P3XQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.604588519 (12h4m50.193933496s) [noise=true , address=104.36.82.56:4249 , piece_id=RD4B7IDIJV25L7EJNSYCZGW5QKJEM435E2N3UFUD44Q4S3DCBOXQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605153860 (12h4m50.194498837s) [noise=true , address=77.90.31.11:4562 , piece_id=XC6F2WLSXKQB7E7MBRNZBJKUSUFL6XNA5TVX5M3ZJI3ZZMK7UBLA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605455377 (12h4m50.194800364s) [noise=true , address=104.36.82.23:4466 , piece_id=5J5X2LSP6GCX2PNRIDZYFJPISWSX3YSRID5M2WU6Y5Z6LWRHLC4Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605743810 (12h4m50.195088797s) [noise=true , address=104.36.82.62:4420 , piece_id=UOO36PYVE5WFRRZLGYLJ6I6SV4EO7W2ZH2TIXJ2OIZZMY3LTHR2A, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.606547457 (12h4m50.195892444s) [noise=true , address=104.36.82.5:4842 , piece_id=E5DSJXN63RFX7EQ4ZTBMUESFW77EB3HQNA5KCQGRVRAA5XO2GN3Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.606982099 (12h4m50.196327086s) [noise=true , address=104.36.82.59:4301 , piece_id=VVR5CQWCPNPZUFEFCPB6ZL2KASWS5XO65FVDI474RKRZSJFDUQTA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.607295849 (12h4m50.196640836s) [noise=true , address=45.91.116.14:4198 , piece_id=2HRU3JZDQSNGD5NJZDLZSRG2WIO2Y2QBSOAY6KUEGUKDBWM53RSA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.607695593 (12h4m50.19704058s) [noise=true , address=104.36.82.6:4881 , piece_id=VUC33D7TLVFICDLA3334DEKBJE5E5P4HQTJU3VDQNMVYWVESSJWA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.608364103 (12h4m50.19770909s) [noise=true , address=104.36.82.2:4757 , piece_id=H6KNYP5HASPWQIHADTMEZDC5AE5HITCAY73XGNWVEXJOPDEZ3SHA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.608764288 (12h4m50.198109265s) [noise=true , address=104.36.82.46:4493 , piece_id=PS4S4OZ6HSITEJFPJDIBWQ5RGWTDRKMX3NCEWXCLC2PSP5Q2KPZQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609065284 (12h4m50.198410271s) [noise=true , address=104.36.82.47:4518 , piece_id=KJ3PJCT3EDH3WZBT7AWWKI6ABGVVAMKZTVJY7Q76FULC72TUWA6Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609312777 (12h4m50.198657755s) [noise=true , address=104.36.82.60:4349 , piece_id=GTTF32YMJWBY2GZEFZPIBZXZWNI7MBLUYCTQ3SRFG735PLWXTXBQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609589888 (12h4m50.198934875s) [noise=true , address=104.36.82.52:4116 , piece_id=7QA2Q5ACN43TQUOKWA34EZFSSQBE3OWNJTYODHHQ5GCVOA4QZOJA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.610295969 (12h4m50.199641056s) [noise=true , address=104.36.82.37:4678 , piece_id=VJO5JO3SJEXBQZC6PJ34DRKWEQKOPI33ZF4IHNNOOTSD7IBRR4PQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.610811305 (12h4m50.200156302s) [noise=true , address=64.49.13.169:4714 , piece_id=A6O3H7QLE6KQTHWUKOEROVHPEHVLG22KNF2KCV3HK2KCLXP7SYVA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:51.715421583 (12h4m51.30476657s) [upload=12552]: Next returned nil segment Aug 7 21:44:51.715438125 (12h4m51.304783112s) [upload=12552]: Waiting for error group managing segments... Aug 7 21:44:51.998131281 (12h4m51.587476268s) [noise=true , address=64.49.13.169:4714 , piece_id=A6O3H7QLE6KQTHWUKOEROVHPEHVLG22KNF2KCV3HK2KCLXP7SYVA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:51.999236335 (12h4m51.588581312s) [noise=true , address=79.127.205.233:4624 , piece_id=WEAYHGF7VE6W5ML3QAE3AITMEJGQFH4JQZ7RQKR7O4NKTPR6RZRA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.638299991 (12h4m52.227644978s) [noise=true , address=45.91.116.6:4002 , piece_id=AE5EL7NWD6FEQ7NXX2JRHB4MVZQDXRGQU3EER2X7COISFAX6P3XQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.639927565 (12h4m52.229272542s) [noise=true , address=104.36.82.6:4881 , piece_id=VUC33D7TLVFICDLA3334DEKBJE5E5P4HQTJU3VDQNMVYWVESSJWA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.643220585 (12h4m52.232565572s) [noise=true , address=104.36.82.2:4757 , piece_id=H6KNYP5HASPWQIHADTMEZDC5AE5HITCAY73XGNWVEXJOPDEZ3SHA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.711872926 (12h4m52.301217913s) [noise=true , address=104.36.82.56:4249 , piece_id=RD4B7IDIJV25L7EJNSYCZGW5QKJEM435E2N3UFUD44Q4S3DCBOXQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.763601896 (12h4m52.352946883s) [noise=true , address=104.36.82.22:4529 , piece_id=ZTTBM5H6MHVMQ3OFVTVISN6WJ7WB3XBWI7YZKUXCFNQ6NO3D4CGA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.768238797 (12h4m52.357583785s) [noise=true , address=104.36.82.60:4349 , piece_id=GTTF32YMJWBY2GZEFZPIBZXZWNI7MBLUYCTQ3SRFG735PLWXTXBQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.798543408 (12h4m52.387888395s) [noise=true , address=45.91.116.14:4198 , piece_id=2HRU3JZDQSNGD5NJZDLZSRG2WIO2Y2QBSOAY6KUEGUKDBWM53RSA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.861192704 (12h4m52.450537691s) [noise=true , address=77.90.31.11:4562 , piece_id=XC6F2WLSXKQB7E7MBRNZBJKUSUFL6XNA5TVX5M3ZJI3ZZMK7UBLA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:53.070218164 (12h4m52.659563151s) [noise=true , address=104.36.82.59:4301 , piece_id=VVR5CQWCPNPZUFEFCPB6ZL2KASWS5XO65FVDI474RKRZSJFDUQTA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:53.558930346 (12h4m53.148275333s) [noise=true , address=104.36.82.46:4493 , piece_id=PS4S4OZ6HSITEJFPJDIBWQ5RGWTDRKMX3NCEWXCLC2PSP5Q2KPZQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.132342777 (12h4m54.721687765s) [noise=true , address=104.36.82.37:4678 , piece_id=VJO5JO3SJEXBQZC6PJ34DRKWEQKOPI33ZF4IHNNOOTSD7IBRR4PQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.265236558 (12h4m54.854581545s) [noise=true , address=104.36.82.52:4116 , piece_id=7QA2Q5ACN43TQUOKWA34EZFSSQBE3OWNJTYODHHQ5GCVOA4QZOJA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.265254161 (12h4m54.854599138s) [seg_pos={0 0}, upload=12552]: Segment reached optimal threshold of 14 pieces. Aug 7 21:44:55.265482308 (12h4m54.854827295s) [noise=true , address=104.36.82.5:4842 , piece_id=E5DSJXN63RFX7EQ4ZTBMUESFW77EB3HQNA5KCQGRVRAA5XO2GN3Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn): protocol: expected piece hash; write tcp 188.166.225.170:57020->104.36.82.5:4842: use of closed network connection; write tcp 188.166.225.170:57020->104.36.82.5:4842: use of closed network connection; EOF Aug 7 21:44:55.265551430 (12h4m54.854896417s) [noise=true , address=104.36.82.47:4518 , piece_id=KJ3PJCT3EDH3WZBT7AWWKI6ABGVVAMKZTVJY7Q76FULC72TUWA6Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17TZ9JLdCivFF46guhSTW6hcoUmTz5uo5wvLkDqC4BGmpm9eLb): protocol: expected piece hash; context canceled; EOF Aug 7 21:44:55.265766391 (12h4m54.855111378s) [noise=true , address=104.36.82.23:4466 , piece_id=5J5X2LSP6GCX2PNRIDZYFJPISWSX3YSRID5M2WU6Y5Z6LWRHLC4Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: write tcp 188.166.225.170:49724->104.36.82.23:4466: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:44:55.265824172 (12h4m54.855169159s) [noise=true , address=104.36.82.62:4420 , piece_id=UOO36PYVE5WFRRZLGYLJ6I6SV4EO7W2ZH2TIXJ2OIZZMY3LTHR2A, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SWVQwL5PxDLaSXADwTrAYS2hVcNBuYAA6ReWuRM3UZa57QyYi): protocol: read tcp 188.166.225.170:33092->104.36.82.62:4420: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:44:55.265833430 (12h4m54.855178417s) [seg_pos={0 0}, upload=12552]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:44:55.265888796 (12h4m54.855233783s) [upload=12552]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:55.265892303 (12h4m54.85523728s) [upload=12552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:46.088275738 (12h8m45.677620725s) [upload=12552]: Done uploading segments... Aug 7 21:48:46.088301307 (12h8m45.677646284s) [upload=12552]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.712627544s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXXXXXXXXXXXXXXXXX..XX..X.] duration: 5.795433732s FAIL (2025-08-07 21:48:52 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.795459241s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 26 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXXXXXXXXXXXXXXX.....X] duration: 4m21.683056411s FAIL (2025-08-07 21:48:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== t segment. Inline: false Aug 7 21:44:25.061589945 (12h4m24.650934922s) [upload=12552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:25.061594724 (12h4m24.650939711s) [upload=12552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:44:50.602885060 (12h4m50.192230047s) [seg_pos={0 0}, upload=12552]: Begin upload segment... Aug 7 21:44:50.602935446 (12h4m50.192280423s) [seg_pos={0 0}, upload=12552]: Done begin upload segment. Aug 7 21:44:50.603112565 (12h4m50.192457552s) [noise=true , address=79.127.205.233:4624 , piece_id=WEAYHGF7VE6W5ML3QAE3AITMEJGQFH4JQZ7RQKR7O4NKTPR6RZRA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.603695610 (12h4m50.193040597s) [noise=true , address=104.36.82.22:4529 , piece_id=ZTTBM5H6MHVMQ3OFVTVISN6WJ7WB3XBWI7YZKUXCFNQ6NO3D4CGA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.604300768 (12h4m50.193645755s) [noise=true , address=45.91.116.6:4002 , piece_id=AE5EL7NWD6FEQ7NXX2JRHB4MVZQDXRGQU3EER2X7COISFAX6P3XQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.604588519 (12h4m50.193933496s) [noise=true , address=104.36.82.56:4249 , piece_id=RD4B7IDIJV25L7EJNSYCZGW5QKJEM435E2N3UFUD44Q4S3DCBOXQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605153860 (12h4m50.194498837s) [noise=true , address=77.90.31.11:4562 , piece_id=XC6F2WLSXKQB7E7MBRNZBJKUSUFL6XNA5TVX5M3ZJI3ZZMK7UBLA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605455377 (12h4m50.194800364s) [noise=true , address=104.36.82.23:4466 , piece_id=5J5X2LSP6GCX2PNRIDZYFJPISWSX3YSRID5M2WU6Y5Z6LWRHLC4Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.605743810 (12h4m50.195088797s) [noise=true , address=104.36.82.62:4420 , piece_id=UOO36PYVE5WFRRZLGYLJ6I6SV4EO7W2ZH2TIXJ2OIZZMY3LTHR2A, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.606547457 (12h4m50.195892444s) [noise=true , address=104.36.82.5:4842 , piece_id=E5DSJXN63RFX7EQ4ZTBMUESFW77EB3HQNA5KCQGRVRAA5XO2GN3Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.606982099 (12h4m50.196327086s) [noise=true , address=104.36.82.59:4301 , piece_id=VVR5CQWCPNPZUFEFCPB6ZL2KASWS5XO65FVDI474RKRZSJFDUQTA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.607295849 (12h4m50.196640836s) [noise=true , address=45.91.116.14:4198 , piece_id=2HRU3JZDQSNGD5NJZDLZSRG2WIO2Y2QBSOAY6KUEGUKDBWM53RSA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.607695593 (12h4m50.19704058s) [noise=true , address=104.36.82.6:4881 , piece_id=VUC33D7TLVFICDLA3334DEKBJE5E5P4HQTJU3VDQNMVYWVESSJWA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.608364103 (12h4m50.19770909s) [noise=true , address=104.36.82.2:4757 , piece_id=H6KNYP5HASPWQIHADTMEZDC5AE5HITCAY73XGNWVEXJOPDEZ3SHA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.608764288 (12h4m50.198109265s) [noise=true , address=104.36.82.46:4493 , piece_id=PS4S4OZ6HSITEJFPJDIBWQ5RGWTDRKMX3NCEWXCLC2PSP5Q2KPZQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609065284 (12h4m50.198410271s) [noise=true , address=104.36.82.47:4518 , piece_id=KJ3PJCT3EDH3WZBT7AWWKI6ABGVVAMKZTVJY7Q76FULC72TUWA6Q, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609312777 (12h4m50.198657755s) [noise=true , address=104.36.82.60:4349 , piece_id=GTTF32YMJWBY2GZEFZPIBZXZWNI7MBLUYCTQ3SRFG735PLWXTXBQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.609589888 (12h4m50.198934875s) [noise=true , address=104.36.82.52:4116 , piece_id=7QA2Q5ACN43TQUOKWA34EZFSSQBE3OWNJTYODHHQ5GCVOA4QZOJA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.610295969 (12h4m50.199641056s) [noise=true , address=104.36.82.37:4678 , piece_id=VJO5JO3SJEXBQZC6PJ34DRKWEQKOPI33ZF4IHNNOOTSD7IBRR4PQ, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:50.610811305 (12h4m50.200156302s) [noise=true , address=64.49.13.169:4714 , piece_id=A6O3H7QLE6KQTHWUKOEROVHPEHVLG22KNF2KCV3HK2KCLXP7SYVA, seg_pos={0 0}, upload=12552]: Uploading piece... Aug 7 21:44:51.715421583 (12h4m51.30476657s) [upload=12552]: Next returned nil segment Aug 7 21:44:51.715438125 (12h4m51.304783112s) [upload=12552]: Waiting for error group managing segments... Aug 7 21:44:51.998131281 (12h4m51.587476268s) [noise=true , address=64.49.13.169:4714 , piece_id=A6O3H7QLE6KQTHWUKOEROVHPEHVLG22KNF2KCV3HK2KCLXP7SYVA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:51.999236335 (12h4m51.588581312s) [noise=true , address=79.127.205.233:4624 , piece_id=WEAYHGF7VE6W5ML3QAE3AITMEJGQFH4JQZ7RQKR7O4NKTPR6RZRA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.638299991 (12h4m52.227644978s) [noise=true , address=45.91.116.6:4002 , piece_id=AE5EL7NWD6FEQ7NXX2JRHB4MVZQDXRGQU3EER2X7COISFAX6P3XQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.639927565 (12h4m52.229272542s) [noise=true , address=104.36.82.6:4881 , piece_id=VUC33D7TLVFICDLA3334DEKBJE5E5P4HQTJU3VDQNMVYWVESSJWA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.643220585 (12h4m52.232565572s) [noise=true , address=104.36.82.2:4757 , piece_id=H6KNYP5HASPWQIHADTMEZDC5AE5HITCAY73XGNWVEXJOPDEZ3SHA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.711872926 (12h4m52.301217913s) [noise=true , address=104.36.82.56:4249 , piece_id=RD4B7IDIJV25L7EJNSYCZGW5QKJEM435E2N3UFUD44Q4S3DCBOXQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.763601896 (12h4m52.352946883s) [noise=true , address=104.36.82.22:4529 , piece_id=ZTTBM5H6MHVMQ3OFVTVISN6WJ7WB3XBWI7YZKUXCFNQ6NO3D4CGA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.768238797 (12h4m52.357583785s) [noise=true , address=104.36.82.60:4349 , piece_id=GTTF32YMJWBY2GZEFZPIBZXZWNI7MBLUYCTQ3SRFG735PLWXTXBQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.798543408 (12h4m52.387888395s) [noise=true , address=45.91.116.14:4198 , piece_id=2HRU3JZDQSNGD5NJZDLZSRG2WIO2Y2QBSOAY6KUEGUKDBWM53RSA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:52.861192704 (12h4m52.450537691s) [noise=true , address=77.90.31.11:4562 , piece_id=XC6F2WLSXKQB7E7MBRNZBJKUSUFL6XNA5TVX5M3ZJI3ZZMK7UBLA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:53.070218164 (12h4m52.659563151s) [noise=true , address=104.36.82.59:4301 , piece_id=VVR5CQWCPNPZUFEFCPB6ZL2KASWS5XO65FVDI474RKRZSJFDUQTA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:53.558930346 (12h4m53.148275333s) [noise=true , address=104.36.82.46:4493 , piece_id=PS4S4OZ6HSITEJFPJDIBWQ5RGWTDRKMX3NCEWXCLC2PSP5Q2KPZQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.132342777 (12h4m54.721687765s) [noise=true , address=104.36.82.37:4678 , piece_id=VJO5JO3SJEXBQZC6PJ34DRKWEQKOPI33ZF4IHNNOOTSD7IBRR4PQ, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.265236558 (12h4m54.854581545s) [noise=true , address=104.36.82.52:4116 , piece_id=7QA2Q5ACN43TQUOKWA34EZFSSQBE3OWNJTYODHHQ5GCVOA4QZOJA, seg_pos={0 0}, upload=12552]: Done uploading piece. err: <nil> Aug 7 21:44:55.265254161 (12h4m54.854599138s) [seg_pos={0 0}, upload=12552]: Segment reached optimal threshold of 14 pieces. Aug 7 21:44:55.265482308 (12h4m54.854827295s) [noise=true , address=104.36.82.5:4842 , piece_id=E5DSJXN63RFX7EQ4ZTBMUESFW77EB3HQNA5KCQGRVRAA5XO2GN3Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn): protocol: expected piece hash; write tcp 188.166.225.170:57020->104.36.82.5:4842: use of closed network connection; write tcp 188.166.225.170:57020->104.36.82.5:4842: use of closed network connection; EOF Aug 7 21:44:55.265551430 (12h4m54.854896417s) [noise=true , address=104.36.82.47:4518 , piece_id=KJ3PJCT3EDH3WZBT7AWWKI6ABGVVAMKZTVJY7Q76FULC72TUWA6Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17TZ9JLdCivFF46guhSTW6hcoUmTz5uo5wvLkDqC4BGmpm9eLb): protocol: expected piece hash; context canceled; EOF Aug 7 21:44:55.265766391 (12h4m54.855111378s) [noise=true , address=104.36.82.23:4466 , piece_id=5J5X2LSP6GCX2PNRIDZYFJPISWSX3YSRID5M2WU6Y5Z6LWRHLC4Q, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rkNnk9RUhKBJ3EsakqMRb3wYwrhRLhB4ZYbrdfKPDk8erRJpP): protocol: write tcp 188.166.225.170:49724->104.36.82.23:4466: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:44:55.265824172 (12h4m54.855169159s) [noise=true , address=104.36.82.62:4420 , piece_id=UOO36PYVE5WFRRZLGYLJ6I6SV4EO7W2ZH2TIXJ2OIZZMY3LTHR2A, seg_pos={0 0}, upload=12552]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SWVQwL5PxDLaSXADwTrAYS2hVcNBuYAA6ReWuRM3UZa57QyYi): protocol: read tcp 188.166.225.170:33092->104.36.82.62:4420: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:44:55.265833430 (12h4m54.855178417s) [seg_pos={0 0}, upload=12552]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:44:55.265888796 (12h4m54.855233783s) [upload=12552]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:55.265892303 (12h4m54.85523728s) [upload=12552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:46.088275738 (12h8m45.677620725s) [upload=12552]: Done uploading segments... Aug 7 21:48:46.088301307 (12h8m45.677646284s) [upload=12552]: Upload finished. err: metaclient: context deadline exceeded Duration: 4m21.683075798s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [..............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.......X.] duration: 6.184351611s FAIL (2025-08-07 21:48:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.184373664s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [...............X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.X..] duration: 6.879311262s FAIL (2025-08-07 21:48:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection Aug 7 21:44:51.028127350 (12h4m50.617472327s) [seg_pos={0 0}, upload=12555]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:44:51.028226048 (12h4m50.617571035s) [upload=12555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:51.028237751 (12h4m50.617582738s) [upload=12555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:51.322864709 (12h4m50.912209697s) [upload=12555]: Done uploading segments... Aug 7 21:44:51.322882523 (12h4m50.9122275s) [upload=12555]: Upload finished. err: <nil> Aug 7 21:45:58.815791269 (12h5m58.405136256s) [upload=12571]: Starting upload Aug 7 21:45:58.815846996 (12h5m58.405191983s) [upload=12571]: Done starting upload Aug 7 21:45:58.815953540 (12h5m58.405298527s) [upload=12571]: Uploading segments... Aug 7 21:45:58.816065333 (12h5m58.405410321s) [upload=12571]: Got next segment. Inline: false Aug 7 21:45:58.816100661 (12h5m58.405445638s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.816106212 (12h5m58.405451199s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:59.064845605 (12h5m58.654190592s) [seg_pos={0 0}, upload=12571]: Begin upload segment... Aug 7 21:45:59.064885521 (12h5m58.654230508s) [seg_pos={0 0}, upload=12571]: Done begin upload segment. Aug 7 21:45:59.064910399 (12h5m58.654255376s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.064952560 (12h5m58.654297547s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065245920 (12h5m58.654590907s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065360070 (12h5m58.654705057s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065517019 (12h5m58.654861996s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065615889 (12h5m58.654960876s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065778009 (12h5m58.655122986s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065884293 (12h5m58.65522928s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066024761 (12h5m58.655369748s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066243109 (12h5m58.655588096s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066452850 (12h5m58.655797837s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066669685 (12h5m58.656014672s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066955041 (12h5m58.656300028s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067149393 (12h5m58.65649438s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067334066 (12h5m58.656679053s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067527215 (12h5m58.656872202s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:46:00.855806043 (12h6m0.44515103s) [upload=12571]: Next returned nil segment Aug 7 21:46:00.855821061 (12h6m0.445166048s) [upload=12571]: Waiting for error group managing segments... Aug 7 21:46:01.180369497 (12h6m0.769714484s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.200492297 (12h6m0.789837284s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.210262691 (12h6m0.799607678s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.220625478 (12h6m0.809970465s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.304447281 (12h6m0.893792308s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.374462398 (12h6m0.963807385s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.378351098 (12h6m0.967696085s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.395257154 (12h6m0.984602141s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.404312138 (12h6m0.993657125s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.409522908 (12h6m0.998867885s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.416315134 (12h6m1.005660121s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.421432725 (12h6m1.010777712s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439709112 (12h6m1.029054099s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439735472 (12h6m1.029080459s) [seg_pos={0 0}, upload=12571]: Segment reached optimal threshold of 13 pieces. Aug 7 21:46:01.440013234 (12h6m1.029358221s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440020518 (12h6m1.029365505s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: write tcp 188.166.225.170:54068->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:46:01.440109398 (12h6m1.029454385s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440122312 (12h6m1.029467299s) [seg_pos={0 0}, upload=12571]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:46:01.440193058 (12h6m1.029538045s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:46:01.440197016 (12h6m1.029542003s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:14.557737130 (12h8m14.147082107s) [upload=12571]: Done uploading segments... Aug 7 21:48:14.557758951 (12h8m14.147103938s) [upload=12571]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.879335841s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [...............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.......X] duration: 30.001009079s FAIL (2025-08-07 21:48:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001031019s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [.............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X..........] duration: 2.50857473s FAIL (2025-08-07 21:48:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.508592635s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [.........................XXXXXXXXXXXXXXXXXXXXXXXXXX...X..X..] duration: 5.549149746s FAIL (2025-08-07 21:48:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.549173761s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................X...............X..........X........] duration: 1.825070848s FAIL (2025-08-07 21:48:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 735125827s) [upload=12376]: Done uploading segments... Aug 7 21:34:02.145826196 (11h54m1.735171183s) [upload=12376]: Upload finished. err: <nil> Aug 7 21:34:47.873603965 (11h54m47.462948952s) [upload=12391]: Starting upload Aug 7 21:34:47.873650053 (11h54m47.46299503s) [upload=12391]: Done starting upload Aug 7 21:34:47.873700940 (11h54m47.463045928s) [upload=12391]: Uploading segments... Aug 7 21:34:47.873733743 (11h54m47.46307872s) [upload=12391]: Got next segment. Inline: true Aug 7 21:34:47.873739335 (11h54m47.463084322s) [upload=12391]: Waiting for error group managing segments... Aug 7 21:34:47.873774331 (11h54m47.463119318s) [upload=12391]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:47.873777988 (11h54m47.463122975s) [upload=12391]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:47.873780623 (11h54m47.46312561s) [upload=12391]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.144972113 (11h54m47.7343171s) [upload=12391]: Done uploading segments... Aug 7 21:34:48.144991410 (11h54m47.734336397s) [upload=12391]: Upload finished. err: <nil> Aug 7 21:35:20.363321680 (11h55m19.952666667s) [upload=12399]: Starting upload Aug 7 21:35:20.363369802 (11h55m19.952714789s) [upload=12399]: Done starting upload Aug 7 21:35:20.363434526 (11h55m19.952779513s) [upload=12399]: Uploading segments... Aug 7 21:35:20.363500943 (11h55m19.95284597s) [upload=12399]: Got next segment. Inline: true Aug 7 21:35:20.363506363 (11h55m19.95285135s) [upload=12399]: Waiting for error group managing segments... Aug 7 21:35:20.363545789 (11h55m19.952890776s) [upload=12399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:20.363550498 (11h55m19.952895485s) [upload=12399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:20.363552953 (11h55m19.95289793s) [upload=12399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:20.648472420 (11h55m20.237817407s) [upload=12399]: Done uploading segments... Aug 7 21:35:20.648492398 (11h55m20.237837386s) [upload=12399]: Upload finished. err: <nil> Aug 7 21:35:35.916328511 (11h55m35.505673488s) [upload=12404]: Starting upload Aug 7 21:35:35.916377014 (11h55m35.505722001s) [upload=12404]: Done starting upload Aug 7 21:35:35.916425818 (11h55m35.505770805s) [upload=12404]: Uploading segments... Aug 7 21:35:35.916457979 (11h55m35.505802956s) [upload=12404]: Got next segment. Inline: true Aug 7 21:35:35.916461466 (11h55m35.505806443s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:35:35.916496753 (11h55m35.50584173s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:35.916501372 (11h55m35.505846359s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:35.916503666 (11h55m35.505848653s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:36.531148061 (11h55m36.120493049s) [upload=12404]: Done uploading segments... Aug 7 21:35:36.531167528 (11h55m36.120512505s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:36:48.991080864 (11h56m48.580425851s) [upload=12421]: Starting upload Aug 7 21:36:48.991131251 (11h56m48.580476238s) [upload=12421]: Done starting upload Aug 7 21:36:48.991177239 (11h56m48.580522226s) [upload=12421]: Uploading segments... Aug 7 21:36:48.991205413 (11h56m48.58055039s) [upload=12421]: Got next segment. Inline: true Aug 7 21:36:48.991209060 (11h56m48.580554037s) [upload=12421]: Waiting for error group managing segments... Aug 7 21:36:48.991242914 (11h56m48.580587901s) [upload=12421]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:48.991246341 (11h56m48.580591328s) [upload=12421]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:48.991248505 (11h56m48.580593492s) [upload=12421]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:49.270270137 (11h56m48.859615134s) [upload=12421]: Done uploading segments... Aug 7 21:36:49.270288603 (11h56m48.85963359s) [upload=12421]: Upload finished. err: <nil> Aug 7 21:37:34.116423494 (11h57m33.705768481s) [upload=12435]: Starting upload Aug 7 21:37:34.116478008 (11h57m33.705822996s) [upload=12435]: Done starting upload Aug 7 21:37:34.116546550 (11h57m33.705891537s) [upload=12435]: Uploading segments... Aug 7 21:37:34.116585414 (11h57m33.705930401s) [upload=12435]: Got next segment. Inline: true Aug 7 21:37:34.116589281 (11h57m33.705934268s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:37:34.116631473 (11h57m33.70597646s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:34.116635470 (11h57m33.705980457s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:34.116638065 (11h57m33.705983042s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:34.430160454 (11h57m34.019505441s) [upload=12435]: Done uploading segments... Aug 7 21:37:34.430179780 (11h57m34.019524767s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:38:07.899363525 (11h58m7.488708512s) [upload=12444]: Starting upload Aug 7 21:38:07.899432257 (11h58m7.488777254s) [upload=12444]: Done starting upload Aug 7 21:38:07.899478635 (11h58m7.488823622s) [upload=12444]: Uploading segments... Aug 7 21:38:07.899513011 (11h58m7.488857998s) [upload=12444]: Got next segment. Inline: true Aug 7 21:38:07.899516348 (11h58m7.488861325s) [upload=12444]: Waiting for error group managing segments... Aug 7 21:38:07.899544021 (11h58m7.488888998s) [upload=12444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:07.899547738 (11h58m7.488892725s) [upload=12444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:07.899551905 (11h58m7.488896892s) [upload=12444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:08.175487142 (11h58m7.764832129s) [upload=12444]: Done uploading segments... Aug 7 21:38:08.175506650 (11h58m7.764851637s) [upload=12444]: Upload finished. err: <nil> Aug 7 21:39:12.808926452 (11h59m12.398271439s) [upload=12459]: Starting upload Aug 7 21:39:12.808989873 (11h59m12.39833486s) [upload=12459]: Done starting upload Aug 7 21:39:12.809038286 (11h59m12.398383273s) [upload=12459]: Uploading segments... Aug 7 21:39:12.809068093 (11h59m12.39841308s) [upload=12459]: Got next segment. Inline: true Aug 7 21:39:12.809072080 (11h59m12.398417068s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:39:12.809118720 (11h59m12.398463757s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:12.809125252 (11h59m12.398470239s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:12.809127897 (11h59m12.398472884s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:13.234618442 (11h59m12.823963429s) [upload=12459]: Done uploading segments... Aug 7 21:39:13.234636597 (11h59m12.823981584s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:39:55.593720326 (11h59m55.183065313s) [upload=12472]: Starting upload Aug 7 21:39:55.593767186 (11h59m55.183112163s) [upload=12472]: Done starting upload Aug 7 21:39:55.593842129 (11h59m55.183187117s) [upload=12472]: Uploading segments... Aug 7 21:39:55.593879050 (11h59m55.183224037s) [upload=12472]: Got next segment. Inline: true Aug 7 21:39:55.593882546 (11h59m55.183227533s) [upload=12472]: Waiting for error group managing segments... Aug 7 21:39:55.593915990 (11h59m55.183260978s) [upload=12472]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:55.593919928 (11h59m55.183264916s) [upload=12472]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:55.593922323 (11h59m55.18326731s) [upload=12472]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:55.991096020 (11h59m55.580441007s) [upload=12472]: Done uploading segments... Aug 7 21:39:55.991111470 (11h59m55.580456457s) [upload=12472]: Upload finished. err: <nil> Aug 7 21:40:19.545259354 (12h0m19.134604341s) [upload=12480]: Starting upload Aug 7 21:40:19.545304791 (12h0m19.134649778s) [upload=12480]: Done starting upload Aug 7 21:40:19.545361891 (12h0m19.134706878s) [upload=12480]: Uploading segments... Aug 7 21:40:19.545393461 (12h0m19.134738448s) [upload=12480]: Got next segment. Inline: true Aug 7 21:40:19.545397098 (12h0m19.134742085s) [upload=12480]: Waiting for error group managing segments... Aug 7 21:40:19.545429900 (12h0m19.134774887s) [upload=12480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:19.545433667 (12h0m19.134778655s) [upload=12480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:19.545435852 (12h0m19.134780839s) [upload=12480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:19.891464235 (12h0m19.480809212s) [upload=12480]: Done uploading segments... Aug 7 21:40:19.891480988 (12h0m19.480825965s) [upload=12480]: Upload finished. err: <nil> Aug 7 21:41:23.105250182 (12h1m22.694595169s) [upload=12497]: Starting upload Aug 7 21:41:23.105299878 (12h1m22.694644865s) [upload=12497]: Done starting upload Aug 7 21:41:23.105354182 (12h1m22.694699169s) [upload=12497]: Uploading segments... Aug 7 21:41:23.105392174 (12h1m22.694737161s) [upload=12497]: Got next segment. Inline: true Aug 7 21:41:23.105395991 (12h1m22.694740978s) [upload=12497]: Waiting for error group managing segments... Aug 7 21:41:23.105430738 (12h1m22.694775715s) [upload=12497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:23.105434485 (12h1m22.694779472s) [upload=12497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:23.105438433 (12h1m22.69478342s) [upload=12497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:23.395695219 (12h1m22.985040206s) [upload=12497]: Done uploading segments... Aug 7 21:41:23.395713514 (12h1m22.985058501s) [upload=12497]: Upload finished. err: <nil> Duration: 1.825118589s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 33 out of 60 is failed (threshold=3) [...............X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X.......] duration: 1.894776393s FAIL (2025-08-07 21:48:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.894809726s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [.................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X......] duration: 1.890504459s FAIL (2025-08-07 21:48:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.890551568s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.X.] duration: 7.285966136s FAIL (2025-08-07 21:48:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection Aug 7 21:44:51.028127350 (12h4m50.617472327s) [seg_pos={0 0}, upload=12555]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:44:51.028226048 (12h4m50.617571035s) [upload=12555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:51.028237751 (12h4m50.617582738s) [upload=12555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:51.322864709 (12h4m50.912209697s) [upload=12555]: Done uploading segments... Aug 7 21:44:51.322882523 (12h4m50.9122275s) [upload=12555]: Upload finished. err: <nil> Aug 7 21:45:58.815791269 (12h5m58.405136256s) [upload=12571]: Starting upload Aug 7 21:45:58.815846996 (12h5m58.405191983s) [upload=12571]: Done starting upload Aug 7 21:45:58.815953540 (12h5m58.405298527s) [upload=12571]: Uploading segments... Aug 7 21:45:58.816065333 (12h5m58.405410321s) [upload=12571]: Got next segment. Inline: false Aug 7 21:45:58.816100661 (12h5m58.405445638s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.816106212 (12h5m58.405451199s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:59.064845605 (12h5m58.654190592s) [seg_pos={0 0}, upload=12571]: Begin upload segment... Aug 7 21:45:59.064885521 (12h5m58.654230508s) [seg_pos={0 0}, upload=12571]: Done begin upload segment. Aug 7 21:45:59.064910399 (12h5m58.654255376s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.064952560 (12h5m58.654297547s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065245920 (12h5m58.654590907s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065360070 (12h5m58.654705057s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065517019 (12h5m58.654861996s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065615889 (12h5m58.654960876s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065778009 (12h5m58.655122986s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065884293 (12h5m58.65522928s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066024761 (12h5m58.655369748s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066243109 (12h5m58.655588096s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066452850 (12h5m58.655797837s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066669685 (12h5m58.656014672s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066955041 (12h5m58.656300028s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067149393 (12h5m58.65649438s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067334066 (12h5m58.656679053s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067527215 (12h5m58.656872202s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:46:00.855806043 (12h6m0.44515103s) [upload=12571]: Next returned nil segment Aug 7 21:46:00.855821061 (12h6m0.445166048s) [upload=12571]: Waiting for error group managing segments... Aug 7 21:46:01.180369497 (12h6m0.769714484s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.200492297 (12h6m0.789837284s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.210262691 (12h6m0.799607678s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.220625478 (12h6m0.809970465s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.304447281 (12h6m0.893792308s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.374462398 (12h6m0.963807385s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.378351098 (12h6m0.967696085s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.395257154 (12h6m0.984602141s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.404312138 (12h6m0.993657125s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.409522908 (12h6m0.998867885s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.416315134 (12h6m1.005660121s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.421432725 (12h6m1.010777712s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439709112 (12h6m1.029054099s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439735472 (12h6m1.029080459s) [seg_pos={0 0}, upload=12571]: Segment reached optimal threshold of 13 pieces. Aug 7 21:46:01.440013234 (12h6m1.029358221s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440020518 (12h6m1.029365505s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: write tcp 188.166.225.170:54068->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:46:01.440109398 (12h6m1.029454385s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440122312 (12h6m1.029467299s) [seg_pos={0 0}, upload=12571]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:46:01.440193058 (12h6m1.029538045s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:46:01.440197016 (12h6m1.029542003s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:14.557737130 (12h8m14.147082107s) [upload=12571]: Done uploading segments... Aug 7 21:48:14.557758951 (12h8m14.147103938s) [upload=12571]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.285983971s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X.........X......X..X.............X..X...........] duration: 7.140709402s FAIL (2025-08-07 21:48:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== ad=12478]: Done starting upload Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments... Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment... Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment. Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments... Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces. Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments... Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil> Duration: 7.140731204s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [...........................X....X.X...X.....................] duration: 703.717747ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:48:16 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 703.759767ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXXXXXXXXXXXXXXXXX...X..X.] duration: 1m0.001118903s FAIL (2025-08-07 21:48:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001129111s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [..............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.........] duration: 2.405267044s FAIL (2025-08-07 21:48:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.405288536s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 38 out of 60 is failed (threshold=3) [.................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.X] duration: 2m16.391021934s FAIL (2025-08-07 21:48:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46242->104.36.82.11:4115: use of closed network connection Aug 7 21:44:51.028127350 (12h4m50.617472327s) [seg_pos={0 0}, upload=12555]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:44:51.028226048 (12h4m50.617571035s) [upload=12555]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:44:51.028237751 (12h4m50.617582738s) [upload=12555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:51.322864709 (12h4m50.912209697s) [upload=12555]: Done uploading segments... Aug 7 21:44:51.322882523 (12h4m50.9122275s) [upload=12555]: Upload finished. err: <nil> Aug 7 21:45:58.815791269 (12h5m58.405136256s) [upload=12571]: Starting upload Aug 7 21:45:58.815846996 (12h5m58.405191983s) [upload=12571]: Done starting upload Aug 7 21:45:58.815953540 (12h5m58.405298527s) [upload=12571]: Uploading segments... Aug 7 21:45:58.816065333 (12h5m58.405410321s) [upload=12571]: Got next segment. Inline: false Aug 7 21:45:58.816100661 (12h5m58.405445638s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.816106212 (12h5m58.405451199s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:45:59.064845605 (12h5m58.654190592s) [seg_pos={0 0}, upload=12571]: Begin upload segment... Aug 7 21:45:59.064885521 (12h5m58.654230508s) [seg_pos={0 0}, upload=12571]: Done begin upload segment. Aug 7 21:45:59.064910399 (12h5m58.654255376s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.064952560 (12h5m58.654297547s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065245920 (12h5m58.654590907s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065360070 (12h5m58.654705057s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065517019 (12h5m58.654861996s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065615889 (12h5m58.654960876s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065778009 (12h5m58.655122986s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.065884293 (12h5m58.65522928s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066024761 (12h5m58.655369748s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066243109 (12h5m58.655588096s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066452850 (12h5m58.655797837s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066669685 (12h5m58.656014672s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.066955041 (12h5m58.656300028s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067149393 (12h5m58.65649438s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067334066 (12h5m58.656679053s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:45:59.067527215 (12h5m58.656872202s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Uploading piece... Aug 7 21:46:00.855806043 (12h6m0.44515103s) [upload=12571]: Next returned nil segment Aug 7 21:46:00.855821061 (12h6m0.445166048s) [upload=12571]: Waiting for error group managing segments... Aug 7 21:46:01.180369497 (12h6m0.769714484s) [noise=true , address=77.90.31.15:4698 , piece_id=6L3IAVPA4PQGSMKOGSKWYG4NLM6TSA2YSIAM6S6MYSDUW26CP5CA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.200492297 (12h6m0.789837284s) [noise=true , address=45.91.116.6:4975 , piece_id=CI7SM56L5KPNQEV52PLIETEBQVH5CFXTKVQOQIZ73YOPEGYC3KOA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.210262691 (12h6m0.799607678s) [noise=true , address=104.36.82.16:4294 , piece_id=KDCYWDEXSTKCHR23CAOKKTYUMRQS5Y4LH6KIJPQFF737WA5TST7Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.220625478 (12h6m0.809970465s) [noise=true , address=104.36.82.7:4909 , piece_id=EKLBHF6P5BXX5VLSLL4QEOJXQXWM7MCOY67W5OON2UHRNMKOQFAA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.304447281 (12h6m0.893792308s) [noise=true , address=104.36.82.11:4115 , piece_id=AH7EPSWFJFX7XDYUQOFCNXEHKQ5AMQX22NY6OFBMMHATCOVH4YJA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.374462398 (12h6m0.963807385s) [noise=true , address=45.91.116.8:4047 , piece_id=TEMT2SH64IMIPQ3GDE5XCF2R2XLY33HUKRFF6KUWEYH7CDVYD4YQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.378351098 (12h6m0.967696085s) [noise=true , address=104.36.82.6:4875 , piece_id=W43EX62262JONZAGLUREOG3EGMICRBIVX5O445J3OQDQ2O475J7A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.395257154 (12h6m0.984602141s) [noise=true , address=45.91.116.6:4973 , piece_id=7MZSPAEXC3Q7W7U3QMZUQRUI3OHPQI3XRAJW4OGU7UZ73FEI4LIQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.404312138 (12h6m0.993657125s) [noise=true , address=77.90.31.19:4843 , piece_id=S73TOGLK5HDCBE4BINZ72A3URDR6TUD43IVIFDY5V37EVCN6222A, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.409522908 (12h6m0.998867885s) [noise=true , address=45.91.116.13:4154 , piece_id=LDSWQ62RRGHTT544S6463WLDKW3WFBWIFQGQDHLYWJJD2ESMYJNQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.416315134 (12h6m1.005660121s) [noise=true , address=45.91.116.12:4227 , piece_id=MDQGJ65RT4QEUFX6OFXPEAOFSTG2VXBRC23DXVVD2IV6V52UFF4Q, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.421432725 (12h6m1.010777712s) [noise=true , address=45.91.116.8:4048 , piece_id=2LZRN76O3QS6FODVI52DIIMK7Z77QQBGIXYUAKZCSFI7WMGRTDUA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439709112 (12h6m1.029054099s) [noise=true , address=45.91.116.9:4262 , piece_id=D4ABUUAX4MHYLFPR6LN2REDC6VRT2W7D4VZHHYM7GZ6HWOWMG2LA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: <nil> Aug 7 21:46:01.439735472 (12h6m1.029080459s) [seg_pos={0 0}, upload=12571]: Segment reached optimal threshold of 13 pieces. Aug 7 21:46:01.440013234 (12h6m1.029358221s) [noise=true , address=77.90.31.11:4554 , piece_id=LMSB62TMAJONFULH4DAOTPWQMGLUFRZYKVG4HNJ4AUBP7EBP5FFQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440020518 (12h6m1.029365505s) [noise=true , address=104.36.82.6:4874 , piece_id=T3W6RCAW4YPEU42CESOSUSFSNPDBFXR74WZUYQ2AHJNBZV776MVQ, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: write tcp 188.166.225.170:54068->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:46:01.440109398 (12h6m1.029454385s) [noise=true , address=77.90.31.18:4807 , piece_id=63E5KXGYEJHGHYE4SKTNTMED5R4HSWIST4CR4EV6ZGON3YHV73AA, seg_pos={0 0}, upload=12571]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 21:46:01.440122312 (12h6m1.029467299s) [seg_pos={0 0}, upload=12571]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:46:01.440193058 (12h6m1.029538045s) [upload=12571]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:46:01.440197016 (12h6m1.029542003s) [upload=12571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:14.557737130 (12h8m14.147082107s) [upload=12571]: Done uploading segments... Aug 7 21:48:14.557758951 (12h8m14.147103938s) [upload=12571]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m16.391043967s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 32 out of 60 is failed (threshold=3) [......................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..X..] duration: 1.925805099s FAIL (2025-08-07 21:48:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 6664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:11.998653609 (12h5m11.587998596s) [upload=12563]: Starting upload Aug 7 21:45:11.998732921 (12h5m11.588077908s) [upload=12563]: Done starting upload Aug 7 21:45:11.998811802 (12h5m11.588156789s) [upload=12563]: Uploading segments... Aug 7 21:45:11.998922443 (12h5m11.58826743s) [upload=12563]: Got next segment. Inline: true Aug 7 21:45:11.998926992 (12h5m11.588271979s) [upload=12563]: Waiting for error group managing segments... Aug 7 21:45:11.998979913 (12h5m11.5883249s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:11.998987267 (12h5m11.588332254s) [upload=12563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:11.998990603 (12h5m11.58833559s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:14.137150454 (12h5m13.726495441s) [upload=12563]: Done uploading segments... Aug 7 21:45:14.137168529 (12h5m13.726513516s) [upload=12563]: Upload finished. err: <nil> Aug 7 21:45:53.851988781 (12h5m53.441333768s) [upload=12569]: Starting upload Aug 7 21:45:53.852043686 (12h5m53.441388673s) [upload=12569]: Done starting upload Aug 7 21:45:53.852088772 (12h5m53.441433759s) [upload=12569]: Uploading segments... Aug 7 21:45:53.852141613 (12h5m53.4414866s) [upload=12569]: Got next segment. Inline: true Aug 7 21:45:53.852146192 (12h5m53.441491169s) [upload=12569]: Waiting for error group managing segments... Aug 7 21:45:53.852182612 (12h5m53.441527599s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:53.852186920 (12h5m53.441531907s) [upload=12569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:53.852189195 (12h5m53.441534182s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:54.127817304 (12h5m53.717162291s) [upload=12569]: Done uploading segments... Aug 7 21:45:54.127834497 (12h5m53.717179484s) [upload=12569]: Upload finished. err: <nil> Aug 7 21:46:54.936546775 (12h6m54.525891762s) [upload=12585]: Starting upload Aug 7 21:46:54.936595597 (12h6m54.525940585s) [upload=12585]: Done starting upload Aug 7 21:46:54.936645253 (12h6m54.52599024s) [upload=12585]: Uploading segments... Aug 7 21:46:54.936675961 (12h6m54.526020948s) [upload=12585]: Got next segment. Inline: true Aug 7 21:46:54.936680381 (12h6m54.526025358s) [upload=12585]: Waiting for error group managing segments... Aug 7 21:46:54.936716089 (12h6m54.526061076s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:54.936720797 (12h6m54.526065774s) [upload=12585]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:54.936723172 (12h6m54.526068159s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:54.288569811 (12h7m53.877914798s) [upload=12585]: Done uploading segments... Aug 7 21:47:54.288620218 (12h7m53.877965205s) [upload=12585]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.925874241s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXXXXXXXXXXXXXXXX..XX..X] duration: 30.000595036s FAIL (2025-08-07 21:48:02 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000615375s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [....................................XX.XX..X.XX.............] duration: 236.566241ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:48:02 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 236.617619ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 33 out of 60 is failed (threshold=3) [................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X......] duration: 1.894948862s FAIL (2025-08-07 21:48:00 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.894976365s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [..................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X.....] duration: 1.925955257s FAIL (2025-08-07 21:47:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.925980314s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [............X.........X......X..X.............X..X..........] duration: 7.325313575s FAIL (2025-08-07 21:47:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== ad=12478]: Done starting upload Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments... Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment... Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment. Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments... Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces. Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments... Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil> Duration: 7.325340205s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 32 out of 60 is failed (threshold=3) [.......................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..X.] duration: 2.110587836s FAIL (2025-08-07 21:47:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 6664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:11.998653609 (12h5m11.587998596s) [upload=12563]: Starting upload Aug 7 21:45:11.998732921 (12h5m11.588077908s) [upload=12563]: Done starting upload Aug 7 21:45:11.998811802 (12h5m11.588156789s) [upload=12563]: Uploading segments... Aug 7 21:45:11.998922443 (12h5m11.58826743s) [upload=12563]: Got next segment. Inline: true Aug 7 21:45:11.998926992 (12h5m11.588271979s) [upload=12563]: Waiting for error group managing segments... Aug 7 21:45:11.998979913 (12h5m11.5883249s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:11.998987267 (12h5m11.588332254s) [upload=12563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:11.998990603 (12h5m11.58833559s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:14.137150454 (12h5m13.726495441s) [upload=12563]: Done uploading segments... Aug 7 21:45:14.137168529 (12h5m13.726513516s) [upload=12563]: Upload finished. err: <nil> Aug 7 21:45:53.851988781 (12h5m53.441333768s) [upload=12569]: Starting upload Aug 7 21:45:53.852043686 (12h5m53.441388673s) [upload=12569]: Done starting upload Aug 7 21:45:53.852088772 (12h5m53.441433759s) [upload=12569]: Uploading segments... Aug 7 21:45:53.852141613 (12h5m53.4414866s) [upload=12569]: Got next segment. Inline: true Aug 7 21:45:53.852146192 (12h5m53.441491169s) [upload=12569]: Waiting for error group managing segments... Aug 7 21:45:53.852182612 (12h5m53.441527599s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:53.852186920 (12h5m53.441531907s) [upload=12569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:53.852189195 (12h5m53.441534182s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:54.127817304 (12h5m53.717162291s) [upload=12569]: Done uploading segments... Aug 7 21:45:54.127834497 (12h5m53.717179484s) [upload=12569]: Upload finished. err: <nil> Aug 7 21:46:54.936546775 (12h6m54.525891762s) [upload=12585]: Starting upload Aug 7 21:46:54.936595597 (12h6m54.525940585s) [upload=12585]: Done starting upload Aug 7 21:46:54.936645253 (12h6m54.52599024s) [upload=12585]: Uploading segments... Aug 7 21:46:54.936675961 (12h6m54.526020948s) [upload=12585]: Got next segment. Inline: true Aug 7 21:46:54.936680381 (12h6m54.526025358s) [upload=12585]: Waiting for error group managing segments... Aug 7 21:46:54.936716089 (12h6m54.526061076s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:54.936720797 (12h6m54.526065774s) [upload=12585]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:54.936723172 (12h6m54.526068159s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:54.288569811 (12h7m53.877914798s) [upload=12585]: Done uploading segments... Aug 7 21:47:54.288620218 (12h7m53.877965205s) [upload=12585]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.110609958s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 32 out of 60 is failed (threshold=3) [........................X.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XX..X] duration: 1m0.001354065s FAIL (2025-08-07 21:47:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 6664 (11h58m33.293011651s) [upload=12450]: Starting upload Aug 7 21:38:33.703724295 (11h58m33.293069282s) [upload=12450]: Done starting upload Aug 7 21:38:33.703770663 (11h58m33.29311565s) [upload=12450]: Uploading segments... Aug 7 21:38:33.703817212 (11h58m33.293162199s) [upload=12450]: Got next segment. Inline: true Aug 7 21:38:33.703820899 (11h58m33.293165886s) [upload=12450]: Waiting for error group managing segments... Aug 7 21:38:33.703853411 (11h58m33.293198388s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:33.703857378 (11h58m33.293202365s) [upload=12450]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:33.703859744 (11h58m33.293204731s) [upload=12450]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:33.059235349 (11h59m32.648580326s) [upload=12450]: Done uploading segments... Aug 7 21:39:33.059253302 (11h59m32.648598289s) [upload=12450]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:33.956833499 (11h59m33.546178486s) [upload=12464]: Starting upload Aug 7 21:39:33.956907240 (11h59m33.546252227s) [upload=12464]: Done starting upload Aug 7 21:39:33.956967195 (11h59m33.546312182s) [upload=12464]: Uploading segments... Aug 7 21:39:33.957011540 (11h59m33.546356527s) [upload=12464]: Got next segment. Inline: true Aug 7 21:39:33.957016229 (11h59m33.546361226s) [upload=12464]: Waiting for error group managing segments... Aug 7 21:39:33.957053820 (11h59m33.546398807s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:33.957058250 (11h59m33.546403237s) [upload=12464]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:33.957061736 (11h59m33.546406723s) [upload=12464]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:33.302006486 (12h0m32.891351473s) [upload=12464]: Done uploading segments... Aug 7 21:40:33.302025733 (12h0m32.89137071s) [upload=12464]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:34.211725785 (12h0m33.801070772s) [upload=12482]: Starting upload Aug 7 21:40:34.211806970 (12h0m33.801151957s) [upload=12482]: Done starting upload Aug 7 21:40:34.211965303 (12h0m33.8013103s) [upload=12482]: Uploading segments... Aug 7 21:40:34.212066617 (12h0m33.801411614s) [upload=12482]: Got next segment. Inline: true Aug 7 21:40:34.212072819 (12h0m33.801417806s) [upload=12482]: Waiting for error group managing segments... Aug 7 21:40:34.212126081 (12h0m33.801471068s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:34.212132573 (12h0m33.80147756s) [upload=12482]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:34.212136120 (12h0m33.801481107s) [upload=12482]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:33.557664583 (12h1m33.14700957s) [upload=12482]: Done uploading segments... Aug 7 21:41:33.557694410 (12h1m33.147039387s) [upload=12482]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:34.468736652 (12h1m34.058081639s) [upload=12501]: Starting upload Aug 7 21:41:34.468803340 (12h1m34.058148327s) [upload=12501]: Done starting upload Aug 7 21:41:34.468862643 (12h1m34.05820763s) [upload=12501]: Uploading segments... Aug 7 21:41:34.468901758 (12h1m34.058246735s) [upload=12501]: Got next segment. Inline: true Aug 7 21:41:34.468905907 (12h1m34.058250884s) [upload=12501]: Waiting for error group managing segments... Aug 7 21:41:34.468943147 (12h1m34.058288134s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:34.468947105 (12h1m34.058292092s) [upload=12501]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:34.468949690 (12h1m34.058294677s) [upload=12501]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:33.814395640 (12h2m33.403740627s) [upload=12501]: Done uploading segments... Aug 7 21:42:33.814423744 (12h2m33.403768731s) [upload=12501]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:46.083099596 (12h2m45.672444573s) [upload=12522]: Starting upload Aug 7 21:42:46.083172004 (12h2m45.672516981s) [upload=12522]: Done starting upload Aug 7 21:42:46.083231107 (12h2m45.672576094s) [upload=12522]: Uploading segments... Aug 7 21:42:46.083339093 (12h2m45.67268408s) [upload=12522]: Got next segment. Inline: true Aug 7 21:42:46.083344093 (12h2m45.67268908s) [upload=12522]: Waiting for error group managing segments... Aug 7 21:42:46.083408827 (12h2m45.672753814s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:46.083414187 (12h2m45.672759174s) [upload=12522]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:46.083416732 (12h2m45.672761719s) [upload=12522]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:47.411975663 (12h2m47.00132066s) [upload=12522]: Done uploading segments... Aug 7 21:42:47.411996032 (12h2m47.001341019s) [upload=12522]: Upload finished. err: <nil> Aug 7 21:43:11.548517821 (12h3m11.137862808s) [upload=12529]: Starting upload Aug 7 21:43:11.548568578 (12h3m11.137913565s) [upload=12529]: Done starting upload Aug 7 21:43:11.548617301 (12h3m11.137962288s) [upload=12529]: Uploading segments... Aug 7 21:43:11.548646878 (12h3m11.137991865s) [upload=12529]: Got next segment. Inline: true Aug 7 21:43:11.548650374 (12h3m11.137995362s) [upload=12529]: Waiting for error group managing segments... Aug 7 21:43:11.548681634 (12h3m11.138026621s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:11.548685361 (12h3m11.138030338s) [upload=12529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:11.548687726 (12h3m11.138032713s) [upload=12529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:10.906282559 (12h4m10.495627546s) [upload=12529]: Done uploading segments... Aug 7 21:44:10.906303038 (12h4m10.495648015s) [upload=12529]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:11.778510580 (12h4m11.367855567s) [upload=12548]: Starting upload Aug 7 21:44:11.778570013 (12h4m11.367915s) [upload=12548]: Done starting upload Aug 7 21:44:11.778662139 (12h4m11.368007126s) [upload=12548]: Uploading segments... Aug 7 21:44:11.778800213 (12h4m11.36814519s) [upload=12548]: Got next segment. Inline: true Aug 7 21:44:11.778804892 (12h4m11.368149869s) [upload=12548]: Waiting for error group managing segments... Aug 7 21:44:11.778861480 (12h4m11.368206467s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:11.778866230 (12h4m11.368211207s) [upload=12548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:11.778868745 (12h4m11.368213732s) [upload=12548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:11.131015060 (12h5m10.720360047s) [upload=12548]: Done uploading segments... Aug 7 21:45:11.131033725 (12h5m10.720378712s) [upload=12548]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:11.998653609 (12h5m11.587998596s) [upload=12563]: Starting upload Aug 7 21:45:11.998732921 (12h5m11.588077908s) [upload=12563]: Done starting upload Aug 7 21:45:11.998811802 (12h5m11.588156789s) [upload=12563]: Uploading segments... Aug 7 21:45:11.998922443 (12h5m11.58826743s) [upload=12563]: Got next segment. Inline: true Aug 7 21:45:11.998926992 (12h5m11.588271979s) [upload=12563]: Waiting for error group managing segments... Aug 7 21:45:11.998979913 (12h5m11.5883249s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:11.998987267 (12h5m11.588332254s) [upload=12563]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:11.998990603 (12h5m11.58833559s) [upload=12563]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:14.137150454 (12h5m13.726495441s) [upload=12563]: Done uploading segments... Aug 7 21:45:14.137168529 (12h5m13.726513516s) [upload=12563]: Upload finished. err: <nil> Aug 7 21:45:53.851988781 (12h5m53.441333768s) [upload=12569]: Starting upload Aug 7 21:45:53.852043686 (12h5m53.441388673s) [upload=12569]: Done starting upload Aug 7 21:45:53.852088772 (12h5m53.441433759s) [upload=12569]: Uploading segments... Aug 7 21:45:53.852141613 (12h5m53.4414866s) [upload=12569]: Got next segment. Inline: true Aug 7 21:45:53.852146192 (12h5m53.441491169s) [upload=12569]: Waiting for error group managing segments... Aug 7 21:45:53.852182612 (12h5m53.441527599s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:53.852186920 (12h5m53.441531907s) [upload=12569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:53.852189195 (12h5m53.441534182s) [upload=12569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:54.127817304 (12h5m53.717162291s) [upload=12569]: Done uploading segments... Aug 7 21:45:54.127834497 (12h5m53.717179484s) [upload=12569]: Upload finished. err: <nil> Aug 7 21:46:54.936546775 (12h6m54.525891762s) [upload=12585]: Starting upload Aug 7 21:46:54.936595597 (12h6m54.525940585s) [upload=12585]: Done starting upload Aug 7 21:46:54.936645253 (12h6m54.52599024s) [upload=12585]: Uploading segments... Aug 7 21:46:54.936675961 (12h6m54.526020948s) [upload=12585]: Got next segment. Inline: true Aug 7 21:46:54.936680381 (12h6m54.526025358s) [upload=12585]: Waiting for error group managing segments... Aug 7 21:46:54.936716089 (12h6m54.526061076s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:54.936720797 (12h6m54.526065774s) [upload=12585]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:54.936723172 (12h6m54.526068159s) [upload=12585]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:54.288569811 (12h7m53.877914798s) [upload=12585]: Done uploading segments... Aug 7 21:47:54.288620218 (12h7m53.877965205s) [upload=12585]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001371088s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X....] duration: 1.869216131s FAIL (2025-08-07 21:47:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.869259856s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [.....................................XX.XX..X.XX............] duration: 238.155853ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:47:28 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 238.176314ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 36 out of 60 is failed (threshold=3) [...............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X........] duration: 2.387609792s FAIL (2025-08-07 21:47:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.387643537s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................X...............X..........X.......] duration: 1.930520162s FAIL (2025-08-07 21:47:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== 735125827s) [upload=12376]: Done uploading segments... Aug 7 21:34:02.145826196 (11h54m1.735171183s) [upload=12376]: Upload finished. err: <nil> Aug 7 21:34:47.873603965 (11h54m47.462948952s) [upload=12391]: Starting upload Aug 7 21:34:47.873650053 (11h54m47.46299503s) [upload=12391]: Done starting upload Aug 7 21:34:47.873700940 (11h54m47.463045928s) [upload=12391]: Uploading segments... Aug 7 21:34:47.873733743 (11h54m47.46307872s) [upload=12391]: Got next segment. Inline: true Aug 7 21:34:47.873739335 (11h54m47.463084322s) [upload=12391]: Waiting for error group managing segments... Aug 7 21:34:47.873774331 (11h54m47.463119318s) [upload=12391]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:34:47.873777988 (11h54m47.463122975s) [upload=12391]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:34:47.873780623 (11h54m47.46312561s) [upload=12391]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:34:48.144972113 (11h54m47.7343171s) [upload=12391]: Done uploading segments... Aug 7 21:34:48.144991410 (11h54m47.734336397s) [upload=12391]: Upload finished. err: <nil> Aug 7 21:35:20.363321680 (11h55m19.952666667s) [upload=12399]: Starting upload Aug 7 21:35:20.363369802 (11h55m19.952714789s) [upload=12399]: Done starting upload Aug 7 21:35:20.363434526 (11h55m19.952779513s) [upload=12399]: Uploading segments... Aug 7 21:35:20.363500943 (11h55m19.95284597s) [upload=12399]: Got next segment. Inline: true Aug 7 21:35:20.363506363 (11h55m19.95285135s) [upload=12399]: Waiting for error group managing segments... Aug 7 21:35:20.363545789 (11h55m19.952890776s) [upload=12399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:20.363550498 (11h55m19.952895485s) [upload=12399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:20.363552953 (11h55m19.95289793s) [upload=12399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:20.648472420 (11h55m20.237817407s) [upload=12399]: Done uploading segments... Aug 7 21:35:20.648492398 (11h55m20.237837386s) [upload=12399]: Upload finished. err: <nil> Aug 7 21:35:35.916328511 (11h55m35.505673488s) [upload=12404]: Starting upload Aug 7 21:35:35.916377014 (11h55m35.505722001s) [upload=12404]: Done starting upload Aug 7 21:35:35.916425818 (11h55m35.505770805s) [upload=12404]: Uploading segments... Aug 7 21:35:35.916457979 (11h55m35.505802956s) [upload=12404]: Got next segment. Inline: true Aug 7 21:35:35.916461466 (11h55m35.505806443s) [upload=12404]: Waiting for error group managing segments... Aug 7 21:35:35.916496753 (11h55m35.50584173s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:35:35.916501372 (11h55m35.505846359s) [upload=12404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:35:35.916503666 (11h55m35.505848653s) [upload=12404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:35:36.531148061 (11h55m36.120493049s) [upload=12404]: Done uploading segments... Aug 7 21:35:36.531167528 (11h55m36.120512505s) [upload=12404]: Upload finished. err: <nil> Aug 7 21:36:48.991080864 (11h56m48.580425851s) [upload=12421]: Starting upload Aug 7 21:36:48.991131251 (11h56m48.580476238s) [upload=12421]: Done starting upload Aug 7 21:36:48.991177239 (11h56m48.580522226s) [upload=12421]: Uploading segments... Aug 7 21:36:48.991205413 (11h56m48.58055039s) [upload=12421]: Got next segment. Inline: true Aug 7 21:36:48.991209060 (11h56m48.580554037s) [upload=12421]: Waiting for error group managing segments... Aug 7 21:36:48.991242914 (11h56m48.580587901s) [upload=12421]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:36:48.991246341 (11h56m48.580591328s) [upload=12421]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:36:48.991248505 (11h56m48.580593492s) [upload=12421]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:36:49.270270137 (11h56m48.859615134s) [upload=12421]: Done uploading segments... Aug 7 21:36:49.270288603 (11h56m48.85963359s) [upload=12421]: Upload finished. err: <nil> Aug 7 21:37:34.116423494 (11h57m33.705768481s) [upload=12435]: Starting upload Aug 7 21:37:34.116478008 (11h57m33.705822996s) [upload=12435]: Done starting upload Aug 7 21:37:34.116546550 (11h57m33.705891537s) [upload=12435]: Uploading segments... Aug 7 21:37:34.116585414 (11h57m33.705930401s) [upload=12435]: Got next segment. Inline: true Aug 7 21:37:34.116589281 (11h57m33.705934268s) [upload=12435]: Waiting for error group managing segments... Aug 7 21:37:34.116631473 (11h57m33.70597646s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:34.116635470 (11h57m33.705980457s) [upload=12435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:34.116638065 (11h57m33.705983042s) [upload=12435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:34.430160454 (11h57m34.019505441s) [upload=12435]: Done uploading segments... Aug 7 21:37:34.430179780 (11h57m34.019524767s) [upload=12435]: Upload finished. err: <nil> Aug 7 21:38:07.899363525 (11h58m7.488708512s) [upload=12444]: Starting upload Aug 7 21:38:07.899432257 (11h58m7.488777254s) [upload=12444]: Done starting upload Aug 7 21:38:07.899478635 (11h58m7.488823622s) [upload=12444]: Uploading segments... Aug 7 21:38:07.899513011 (11h58m7.488857998s) [upload=12444]: Got next segment. Inline: true Aug 7 21:38:07.899516348 (11h58m7.488861325s) [upload=12444]: Waiting for error group managing segments... Aug 7 21:38:07.899544021 (11h58m7.488888998s) [upload=12444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:07.899547738 (11h58m7.488892725s) [upload=12444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:07.899551905 (11h58m7.488896892s) [upload=12444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:08.175487142 (11h58m7.764832129s) [upload=12444]: Done uploading segments... Aug 7 21:38:08.175506650 (11h58m7.764851637s) [upload=12444]: Upload finished. err: <nil> Aug 7 21:39:12.808926452 (11h59m12.398271439s) [upload=12459]: Starting upload Aug 7 21:39:12.808989873 (11h59m12.39833486s) [upload=12459]: Done starting upload Aug 7 21:39:12.809038286 (11h59m12.398383273s) [upload=12459]: Uploading segments... Aug 7 21:39:12.809068093 (11h59m12.39841308s) [upload=12459]: Got next segment. Inline: true Aug 7 21:39:12.809072080 (11h59m12.398417068s) [upload=12459]: Waiting for error group managing segments... Aug 7 21:39:12.809118720 (11h59m12.398463757s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:12.809125252 (11h59m12.398470239s) [upload=12459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:12.809127897 (11h59m12.398472884s) [upload=12459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:13.234618442 (11h59m12.823963429s) [upload=12459]: Done uploading segments... Aug 7 21:39:13.234636597 (11h59m12.823981584s) [upload=12459]: Upload finished. err: <nil> Aug 7 21:39:55.593720326 (11h59m55.183065313s) [upload=12472]: Starting upload Aug 7 21:39:55.593767186 (11h59m55.183112163s) [upload=12472]: Done starting upload Aug 7 21:39:55.593842129 (11h59m55.183187117s) [upload=12472]: Uploading segments... Aug 7 21:39:55.593879050 (11h59m55.183224037s) [upload=12472]: Got next segment. Inline: true Aug 7 21:39:55.593882546 (11h59m55.183227533s) [upload=12472]: Waiting for error group managing segments... Aug 7 21:39:55.593915990 (11h59m55.183260978s) [upload=12472]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:55.593919928 (11h59m55.183264916s) [upload=12472]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:55.593922323 (11h59m55.18326731s) [upload=12472]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:55.991096020 (11h59m55.580441007s) [upload=12472]: Done uploading segments... Aug 7 21:39:55.991111470 (11h59m55.580456457s) [upload=12472]: Upload finished. err: <nil> Aug 7 21:40:19.545259354 (12h0m19.134604341s) [upload=12480]: Starting upload Aug 7 21:40:19.545304791 (12h0m19.134649778s) [upload=12480]: Done starting upload Aug 7 21:40:19.545361891 (12h0m19.134706878s) [upload=12480]: Uploading segments... Aug 7 21:40:19.545393461 (12h0m19.134738448s) [upload=12480]: Got next segment. Inline: true Aug 7 21:40:19.545397098 (12h0m19.134742085s) [upload=12480]: Waiting for error group managing segments... Aug 7 21:40:19.545429900 (12h0m19.134774887s) [upload=12480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:19.545433667 (12h0m19.134778655s) [upload=12480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:19.545435852 (12h0m19.134780839s) [upload=12480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:19.891464235 (12h0m19.480809212s) [upload=12480]: Done uploading segments... Aug 7 21:40:19.891480988 (12h0m19.480825965s) [upload=12480]: Upload finished. err: <nil> Aug 7 21:41:23.105250182 (12h1m22.694595169s) [upload=12497]: Starting upload Aug 7 21:41:23.105299878 (12h1m22.694644865s) [upload=12497]: Done starting upload Aug 7 21:41:23.105354182 (12h1m22.694699169s) [upload=12497]: Uploading segments... Aug 7 21:41:23.105392174 (12h1m22.694737161s) [upload=12497]: Got next segment. Inline: true Aug 7 21:41:23.105395991 (12h1m22.694740978s) [upload=12497]: Waiting for error group managing segments... Aug 7 21:41:23.105430738 (12h1m22.694775715s) [upload=12497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:23.105434485 (12h1m22.694779472s) [upload=12497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:23.105438433 (12h1m22.69478342s) [upload=12497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:23.395695219 (12h1m22.985040206s) [upload=12497]: Done uploading segments... Aug 7 21:41:23.395713514 (12h1m22.985058501s) [upload=12497]: Upload finished. err: <nil> Duration: 1.930554197s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 37 out of 60 is failed (threshold=3) [................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.......] duration: 6.138920059s FAIL (2025-08-07 21:47:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.138941481s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXXXXXXXXXXXXXXXX...X..X] duration: 1m0.001232067s FAIL (2025-08-07 21:47:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001317309s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............X.........X......X..X.............X..X.........] duration: 8.629000575s FAIL (2025-08-07 21:47:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== ad=12478]: Done starting upload Aug 7 21:40:13.160345841 (12h0m12.749690828s) [upload=12478]: Uploading segments... Aug 7 21:40:13.160465419 (12h0m12.749810406s) [upload=12478]: Got next segment. Inline: false Aug 7 21:40:13.160488423 (12h0m12.74983341s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:13.160492521 (12h0m12.749837508s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:40:14.024449389 (12h0m13.613794386s) [seg_pos={0 0}, upload=12478]: Begin upload segment... Aug 7 21:40:14.024497139 (12h0m13.613842126s) [seg_pos={0 0}, upload=12478]: Done begin upload segment. Aug 7 21:40:14.024520314 (12h0m13.613865301s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024527568 (12h0m13.613872555s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024915139 (12h0m13.614260126s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.024946630 (12h0m13.614291617s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025230222 (12h0m13.614575199s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025320585 (12h0m13.614665572s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025541578 (12h0m13.614886565s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025702406 (12h0m13.615047393s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.025850619 (12h0m13.615195606s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026035403 (12h0m13.61538039s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026287314 (12h0m13.615632301s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026331439 (12h0m13.615676426s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026644838 (12h0m13.615989826s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026679736 (12h0m13.616024713s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.026890909 (12h0m13.616235896s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027132201 (12h0m13.616477178s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027330621 (12h0m13.616675608s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:14.027577493 (12h0m13.61692248s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Uploading piece... Aug 7 21:40:15.144921916 (12h0m14.734266903s) [upload=12478]: Next returned nil segment Aug 7 21:40:15.144943788 (12h0m14.734288775s) [upload=12478]: Waiting for error group managing segments... Aug 7 21:40:15.425370688 (12h0m15.014715665s) [noise=true , address=64.49.13.140:4510 , piece_id=VJJD4RVIWXNIDCZXKEEMBDQD6ZQXPOWS3WKWBAFOOKHBUTJH6CLQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.528965431 (12h0m15.118310418s) [noise=true , address=64.49.13.170:4721 , piece_id=N7YWI7FRIBKFLUF7TQZLWTT3YTVRDGHUPQJQ2GUP3ENNCZO6DXHQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.945504159 (12h0m15.534849146s) [noise=true , address=104.36.82.46:4496 , piece_id=FAAS2ISJ3FABRTBOOUGW77IT6A2325ZKX4CSGMHSBRZWSQ4XNOGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:15.973785008 (12h0m15.563129995s) [noise=true , address=104.36.82.12:4163 , piece_id=HCC7QXBZMB3P33M366DZRH2IWVQQVOT56LAWYLY5MW5RARFSISVQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.002098709 (12h0m15.591443696s) [noise=true , address=104.36.82.52:4109 , piece_id=XMAEWF5LMHRJMQAA6FSX6TCYB4OUJ2GVDZQUFDPVFWGXGRQZCZIA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.030274577 (12h0m15.619619604s) [noise=true , address=104.36.82.23:4449 , piece_id=Y52RDGW6QODSWLLQSKBJPSNPUTHB5OLEJHN3Y4GUQ6OZK5XARCRA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.044682273 (12h0m15.63402725s) [noise=true , address=104.36.82.54:4160 , piece_id=ODKE4XCXDIENJAZZJR7FUE35EM3XNJL3MITY2CFDLZRBBSHQIKXQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.053282688 (12h0m15.642627665s) [noise=true , address=45.91.116.5:4324 , piece_id=F55DGRECWWVUHFXVMHYRYUHSTO262DXLEIAB2UABJRTIDWG4MEMA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.058737404 (12h0m15.648082391s) [noise=true , address=104.36.82.58:4291 , piece_id=246EQHXDOHPPFCKBCQNBTJ46UDD5ZRRPHA7YUUGU22FPJJLKNQ2Q, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.093091522 (12h0m15.682436509s) [noise=true , address=104.36.82.12:4177 , piece_id=IPZF77LYNKRXAEDUDGMCUXT4EG5JSIFP3I7E3VPEIN3WNV7XCAKA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.176548758 (12h0m15.765893735s) [noise=true , address=104.36.82.45:4939 , piece_id=BECNPJEALTYQXO4IBONPU27XA2MOFWIP7ATHDJFMYTIG5OVAOHLA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.186654263 (12h0m15.77599925s) [noise=true , address=104.36.82.5:4859 , piece_id=3KS2DGR4OLOW4ZUVHSMTVE6XIXZQ3VWOBUXVGTCJTLHNIWGVZOEA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.255094056 (12h0m15.844439053s) [noise=true , address=45.91.116.7:4012 , piece_id=FCVPPAJSDQDEMSIP3CJPXF5FQXQ6W3G3CAMR66IOILUY4OQK7P4A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268089419 (12h0m15.857434406s) [noise=true , address=77.90.31.18:4828 , piece_id=GPXCXC2RW6MRDMLAZJXDML4JO62QAZAZLYZP3JV7J6DQR7QEMTBA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: <nil> Aug 7 21:40:16.268110299 (12h0m15.857455286s) [seg_pos={0 0}, upload=12478]: Segment reached optimal threshold of 14 pieces. Aug 7 21:40:16.268265978 (12h0m15.857610965s) [noise=true , address=104.36.82.61:4394 , piece_id=2XK3J33NNGFMQU7YDFG35IZ3JFAHUFQ7C3OIUFSNPOSFXB6KEKPQ, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dSpTqCxeAcL9PVYbP7BhmuKE7papFPc9swB1Fy83WcFDgQbcF): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268327325 (12h0m15.857672302s) [noise=true , address=45.91.116.6:4999 , piece_id=CHC3CKW5P6UDSGH2HRU7TMLEEK7A4Q6USGZXSOWVH7SWDBQ6ASDA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB): protocol: expected piece hash; context canceled; EOF Aug 7 21:40:16.268492010 (12h0m15.857836997s) [noise=true , address=104.36.82.36:4643 , piece_id=VU7DYF2KI6TB74ERJLYE6BDDQQPMDBSMLZD4H6DKNRKW442ICVGA, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zQsZ4qrfBa95hJYTzjWJJzSea3noCnZ7W22V8ViohjrB5opEw): protocol: write tcp 188.166.225.170:42342->104.36.82.36:4643: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268605697 (12h0m15.857950684s) [noise=true , address=104.36.82.15:4283 , piece_id=AXCQRAQ6EYNFFHRDCWO6DFLB76ASUYOXWMPMQZYZHHIN6NF3QS5A, seg_pos={0 0}, upload=12478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ytKjZnmERiZtgJuMJhXG5KFwUQwie27odve4LPE7Pw5hR5ZaU): protocol: write tcp 188.166.225.170:45054->104.36.82.15:4283: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:40:16.268677886 (12h0m15.858022873s) [seg_pos={0 0}, upload=12478]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:40:16.268787846 (12h0m15.858132843s) [upload=12478]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:40:16.268796623 (12h0m15.85814161s) [upload=12478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:17.124513483 (12h0m16.71385847s) [upload=12478]: Done uploading segments... Aug 7 21:40:17.124534484 (12h0m16.713879471s) [upload=12478]: Upload finished. err: <nil> Duration: 8.629028518s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X...X...] duration: 1.900397607s FAIL (2025-08-07 21:47:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line exceeded Aug 7 21:37:22.572699237 (11h57m22.162044224s) [upload=12432]: Starting upload Aug 7 21:37:22.572782937 (11h57m22.162127924s) [upload=12432]: Done starting upload Aug 7 21:37:22.572842260 (11h57m22.162187247s) [upload=12432]: Uploading segments... Aug 7 21:37:22.572885313 (11h57m22.1622303s) [upload=12432]: Got next segment. Inline: true Aug 7 21:37:22.572889591 (11h57m22.162234578s) [upload=12432]: Waiting for error group managing segments... Aug 7 21:37:22.572930138 (11h57m22.162275125s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:37:22.572934296 (11h57m22.162279283s) [upload=12432]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:37:22.572937121 (11h57m22.162282118s) [upload=12432]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:37:51.933954276 (11h57m51.523299253s) [upload=12432]: Done uploading segments... Aug 7 21:37:51.933970927 (11h57m51.523315914s) [upload=12432]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:38:28.745940745 (11h58m28.335285732s) [upload=12448]: Starting upload Aug 7 21:38:28.745997504 (11h58m28.335342491s) [upload=12448]: Done starting upload Aug 7 21:38:28.746040176 (11h58m28.335385163s) [upload=12448]: Uploading segments... Aug 7 21:38:28.746078458 (11h58m28.335423446s) [upload=12448]: Got next segment. Inline: true Aug 7 21:38:28.746081766 (11h58m28.335426743s) [upload=12448]: Waiting for error group managing segments... Aug 7 21:38:28.746111923 (11h58m28.33545691s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:28.746115540 (11h58m28.335460527s) [upload=12448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:28.746118205 (11h58m28.335463192s) [upload=12448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:58.105315693 (11h58m57.694660691s) [upload=12448]: Done uploading segments... Aug 7 21:38:58.105342465 (11h58m57.694687452s) [upload=12448]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:27.137238449 (11h59m26.726583436s) [upload=12462]: Starting upload Aug 7 21:39:27.137294907 (11h59m26.726639894s) [upload=12462]: Done starting upload Aug 7 21:39:27.137337368 (11h59m26.726682355s) [upload=12462]: Uploading segments... Aug 7 21:39:27.137366043 (11h59m26.72671103s) [upload=12462]: Got next segment. Inline: true Aug 7 21:39:27.137370061 (11h59m26.726715048s) [upload=12462]: Waiting for error group managing segments... Aug 7 21:39:27.137401901 (11h59m26.726746879s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:27.137405518 (11h59m26.726750495s) [upload=12462]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:27.137407813 (11h59m26.72675279s) [upload=12462]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:56.495186447 (11h59m56.084531434s) [upload=12462]: Done uploading segments... Aug 7 21:39:56.495206415 (11h59m56.084551402s) [upload=12462]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:01.914615953 (12h0m1.50396094s) [upload=12475]: Starting upload Aug 7 21:40:01.914684295 (12h0m1.504029282s) [upload=12475]: Done starting upload Aug 7 21:40:01.914741163 (12h0m1.50408616s) [upload=12475]: Uploading segments... Aug 7 21:40:01.914783103 (12h0m1.50412809s) [upload=12475]: Got next segment. Inline: true Aug 7 21:40:01.914788143 (12h0m1.504133131s) [upload=12475]: Waiting for error group managing segments... Aug 7 21:40:01.914833950 (12h0m1.504178937s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:01.914840073 (12h0m1.50418507s) [upload=12475]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:01.914846014 (12h0m1.504191011s) [upload=12475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:31.268196986 (12h0m30.857542023s) [upload=12475]: Done uploading segments... Aug 7 21:40:31.268223536 (12h0m30.857568533s) [upload=12475]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:46.248438399 (12h0m45.837783386s) [upload=12487]: Starting upload Aug 7 21:40:46.248491410 (12h0m45.837836387s) [upload=12487]: Done starting upload Aug 7 21:40:46.248540414 (12h0m45.837885401s) [upload=12487]: Uploading segments... Aug 7 21:40:46.248572345 (12h0m45.837917332s) [upload=12487]: Got next segment. Inline: true Aug 7 21:40:46.248576133 (12h0m45.83792112s) [upload=12487]: Waiting for error group managing segments... Aug 7 21:40:46.248613013 (12h0m45.837958s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:46.248616850 (12h0m45.837961837s) [upload=12487]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:46.248619255 (12h0m45.837964242s) [upload=12487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:55.261300379 (12h0m54.850645366s) [upload=12487]: Done uploading segments... Aug 7 21:40:55.261318514 (12h0m54.850663501s) [upload=12487]: Upload finished. err: <nil> Aug 7 21:42:00.373248983 (12h1m59.962594s) [upload=12509]: Starting upload Aug 7 21:42:00.373363813 (12h1m59.96270884s) [upload=12509]: Done starting upload Aug 7 21:42:00.373438045 (12h1m59.962783062s) [upload=12509]: Uploading segments... Aug 7 21:42:00.373501075 (12h1m59.962846083s) [upload=12509]: Got next segment. Inline: true Aug 7 21:42:00.373507809 (12h1m59.962852816s) [upload=12509]: Waiting for error group managing segments... Aug 7 21:42:00.373577642 (12h1m59.962922669s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:00.373584325 (12h1m59.962929342s) [upload=12509]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:00.373590016 (12h1m59.962935053s) [upload=12509]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:29.724044552 (12h2m29.313389539s) [upload=12509]: Done uploading segments... Aug 7 21:42:29.724072415 (12h2m29.313417392s) [upload=12509]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:30.611135457 (12h2m30.200480444s) [upload=12516]: Starting upload Aug 7 21:42:30.611201894 (12h2m30.200546881s) [upload=12516]: Done starting upload Aug 7 21:42:30.611252732 (12h2m30.200597719s) [upload=12516]: Uploading segments... Aug 7 21:42:30.611338045 (12h2m30.200683032s) [upload=12516]: Got next segment. Inline: true Aug 7 21:42:30.611344076 (12h2m30.200689063s) [upload=12516]: Waiting for error group managing segments... Aug 7 21:42:30.611400083 (12h2m30.20074507s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:30.611404922 (12h2m30.200749909s) [upload=12516]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:30.611408178 (12h2m30.200753166s) [upload=12516]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:46.271341803 (12h2m45.860686781s) [upload=12516]: Done uploading segments... Aug 7 21:42:46.271358525 (12h2m45.860703512s) [upload=12516]: Upload finished. err: <nil> Aug 7 21:43:05.160958738 (12h3m4.750303725s) [upload=12528]: Starting upload Aug 7 21:43:05.161064639 (12h3m4.750409626s) [upload=12528]: Done starting upload Aug 7 21:43:05.161177936 (12h3m4.750522953s) [upload=12528]: Uploading segments... Aug 7 21:43:05.161261357 (12h3m4.750606344s) [upload=12528]: Got next segment. Inline: true Aug 7 21:43:05.161266376 (12h3m4.750611353s) [upload=12528]: Waiting for error group managing segments... Aug 7 21:43:05.161306081 (12h3m4.750651069s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:05.161310580 (12h3m4.750655567s) [upload=12528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:05.161312864 (12h3m4.750657841s) [upload=12528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:17.899724589 (12h3m17.489069576s) [upload=12528]: Done uploading segments... Aug 7 21:43:17.899747483 (12h3m17.48909251s) [upload=12528]: Upload finished. err: <nil> Aug 7 21:43:34.894582431 (12h3m34.483927418s) [upload=12537]: Starting upload Aug 7 21:43:34.894647576 (12h3m34.483992564s) [upload=12537]: Done starting upload Aug 7 21:43:34.894699496 (12h3m34.484044483s) [upload=12537]: Uploading segments... Aug 7 21:43:34.894732940 (12h3m34.484077927s) [upload=12537]: Got next segment. Inline: true Aug 7 21:43:34.894736967 (12h3m34.484081954s) [upload=12537]: Waiting for error group managing segments... Aug 7 21:43:34.894770662 (12h3m34.484115649s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:34.894775050 (12h3m34.484120027s) [upload=12537]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:34.894777795 (12h3m34.484122782s) [upload=12537]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:35.191263901 (12h3m34.780608888s) [upload=12537]: Done uploading segments... Aug 7 21:43:35.191280001 (12h3m34.780624978s) [upload=12537]: Upload finished. err: <nil> Aug 7 21:44:27.756622626 (12h4m27.345967613s) [upload=12553]: Starting upload Aug 7 21:44:27.756679294 (12h4m27.346024281s) [upload=12553]: Done starting upload Aug 7 21:44:27.756721205 (12h4m27.346066192s) [upload=12553]: Uploading segments... Aug 7 21:44:27.756751423 (12h4m27.34609641s) [upload=12553]: Got next segment. Inline: true Aug 7 21:44:27.756754779 (12h4m27.346099766s) [upload=12553]: Waiting for error group managing segments... Aug 7 21:44:27.756785678 (12h4m27.346130655s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:27.756789615 (12h4m27.346134602s) [upload=12553]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:27.756791790 (12h4m27.346136777s) [upload=12553]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:57.108386877 (12h4m56.697731864s) [upload=12553]: Done uploading segments... Aug 7 21:44:57.108405372 (12h4m56.697750359s) [upload=12553]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.900419419s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 33 out of 60 is failed (threshold=3) [.................X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...X.....] duration: 1.975887038s FAIL (2025-08-07 21:47:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.975908187s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [............................X....X.X...X....................] duration: 762.980461ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:47:05 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 763.003787ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [......................................XX.XX..X.XX...........] duration: 462.223093ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:46:49 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 462.249983ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [.............................X....X.X...X...................] duration: 753.909906ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:46:44 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 753.936467ms edge gateway vhost us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)