Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.735378621s FAIL (2025-07-25 09:54:32 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.735401325s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 5.645029793s FAIL (2025-07-25 09:54:27 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.645057095s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..................XX...X.........X....X.....................] duration: 1.853091777s FAIL (2025-07-25 09:54:27 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.853119799s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.829289325s FAIL (2025-07-25 09:54:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.829314074s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X..........X....XX..........................] duration: 5.767439457s FAIL (2025-07-25 09:54:12 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 5.767468591s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.X......................] duration: 2.121453454s FAIL (2025-07-25 09:54:05 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.121483271s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 5.875571441s FAIL (2025-07-25 09:54:03 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.875592492s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.694372387s FAIL (2025-07-25 09:54:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.694388629s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 5.702032215s FAIL (2025-07-25 09:53:56 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.702065749s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...................XX...X.........X....X....................] duration: 1.897781903s FAIL (2025-07-25 09:53:48 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.89781658s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.949324711s FAIL (2025-07-25 09:53:45 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.949350912s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.142696042s FAIL (2025-07-25 09:53:41 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 6.142716401s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.820826629s FAIL (2025-07-25 09:53:37 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.820882635s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X..........X....XX.........................] duration: 5.844886895s FAIL (2025-07-25 09:53:16 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 5.84491521s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.52261825s FAIL (2025-07-25 09:53:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.52263913s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.782714949s FAIL (2025-07-25 09:53:07 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.782743805s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 5.72831216s FAIL (2025-07-25 09:53:05 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.728344072s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.177023752s FAIL (2025-07-25 09:53:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.177045953s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.X.....................] duration: 1.939767585s FAIL (2025-07-25 09:53:02 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.939797793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.90789894s FAIL (2025-07-25 09:52:51 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.907919831s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....................XX...X.........X....X...................] duration: 1.82378463s FAIL (2025-07-25 09:52:47 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.823821441s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 5.873070894s FAIL (2025-07-25 09:52:45 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.873098105s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.320397516s FAIL (2025-07-25 09:52:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.320417775s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.155243642s FAIL (2025-07-25 09:52:34 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 2.155262319s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.779064685s FAIL (2025-07-25 09:52:25 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.779100242s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X..........X....XX........................] duration: 5.75773129s FAIL (2025-07-25 09:52:22 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 5.757756517s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 5.869626293s FAIL (2025-07-25 09:52:18 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 5.869651241s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 5.924747928s FAIL (2025-07-25 09:52:11 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.9247743s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.91824118s FAIL (2025-07-25 09:52:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.918269375s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.464107592s FAIL (2025-07-25 09:52:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.464129325s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.X....................] duration: 2.038980403s FAIL (2025-07-25 09:51:54 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.039006333s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.583761027s FAIL (2025-07-25 09:51:52 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.58378886s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.919083369s FAIL (2025-07-25 09:51:52 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.919102896s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....................XX...X.........X....X..................] duration: 1.843115368s FAIL (2025-07-25 09:51:51 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.843159563s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.649140077s FAIL (2025-07-25 09:51:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.649167579s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................X..........X....XX.......................] duration: 5.868572315s FAIL (2025-07-25 09:51:46 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 5.868601831s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.587970919s FAIL (2025-07-25 09:51:28 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.58799771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.X...................] duration: 1.995280568s FAIL (2025-07-25 09:51:27 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.995304404s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.66836764s FAIL (2025-07-25 09:51:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E6B456F9A1, HostID: , api error AccessDenied: Access Denied. Duration: 5.668387538s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.925207907s FAIL (2025-07-25 09:51:15 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.925222517s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 6.122343455s FAIL (2025-07-25 09:51:13 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978190206 (21h57m17.700753236s) [noise=true , address=45.91.116.11:4118 , piece_id=DVDR7JBJL2ISRJKZ6NJO6ZFOQEYJL255CMKP7JIMLQNZ56HE7O4A, seg_pos={0 0}, upload=21083]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:07.978226436 (21h57m17.700789476s) [seg_pos={0 0}, upload=21083]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:33:07.978290728 (21h57m17.700853768s) [upload=21083]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:07.978295116 (21h57m17.700858156s) [upload=21083]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:08.272621008 (21h57m17.995184058s) [upload=21083]: Done uploading segments... Jul 25 09:33:08.272644353 (21h57m17.995207393s) [upload=21083]: Upload finished. err: <nil> Jul 25 09:34:19.155612153 (21h58m28.878175193s) [upload=21099]: Starting upload Jul 25 09:34:19.155671557 (21h58m28.878234597s) [upload=21099]: Done starting upload Jul 25 09:34:19.155788951 (21h58m28.878351992s) [upload=21099]: Uploading segments... Jul 25 09:34:19.155880987 (21h58m28.878444027s) [upload=21099]: Got next segment. Inline: false Jul 25 09:34:19.155911836 (21h58m28.878474876s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:19.155916906 (21h58m28.878479946s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:34:19.474089123 (21h58m29.196652163s) [seg_pos={0 0}, upload=21099]: Begin upload segment... Jul 25 09:34:19.474169006 (21h58m29.196732046s) [seg_pos={0 0}, upload=21099]: Done begin upload segment. Jul 25 09:34:19.474204834 (21h58m29.196767884s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474220154 (21h58m29.196783194s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474645958 (21h58m29.197209008s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.474854367 (21h58m29.197417407s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475043850 (21h58m29.19760689s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475224505 (21h58m29.197787545s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475458443 (21h58m29.198021473s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475717659 (21h58m29.198280699s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.475971265 (21h58m29.198534305s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.476371320 (21h58m29.19893436s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.478271665 (21h58m29.200834705s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479648920 (21h58m29.20221195s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.479910261 (21h58m29.202473321s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.482494515 (21h58m29.205057555s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.483910805 (21h58m29.206473845s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:19.489619007 (21h58m29.212182057s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Uploading piece... Jul 25 09:34:21.166728613 (21h58m30.889291653s) [upload=21099]: Next returned nil segment Jul 25 09:34:21.166748240 (21h58m30.88931128s) [upload=21099]: Waiting for error group managing segments... Jul 25 09:34:21.499554977 (21h58m31.222118017s) [noise=true , address=45.91.116.6:4974 , piece_id=Q56R4JCGQIZQG7ALI2QO3HTN7PQCJCOQEMLW6HCIVOJRT5MEX6KQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.504744947 (21h58m31.227307988s) [noise=true , address=45.91.116.7:4011 , piece_id=SKT2T7L42JDMB5JR74IACUTSRADSRSGBXUAGKWXWVINNVUQAPOUA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.507097449 (21h58m31.229660489s) [noise=true , address=45.91.116.14:4189 , piece_id=6ML7BRXV4AGZBWR523WB3IHVRU7IVHNNHDN6G5ONERT23ODBWQ7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.517304048 (21h58m31.239867088s) [noise=true , address=104.36.82.11:4114 , piece_id=4LNS6F6FDLNPOGMZ4HPIJ44ZP2Q5XTA2T4UJNHIVHBGBOQKYIMSA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.524928355 (21h58m31.247491395s) [noise=true , address=104.36.82.2:4729 , piece_id=Y5KJG5VJCHWRDBLIMSZZUE5ISYOCEZ3Z5FAP4MXTCAFKN2AMAFFQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.533139686 (21h58m31.255702726s) [noise=true , address=104.36.82.6:4875 , piece_id=ICQB7MCL4MEBXDRUXJQR7TWBNJG5FDXK2HC7HCDSLP5D2AKZU2IQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.549770536 (21h58m31.272333576s) [noise=true , address=104.36.82.9:4005 , piece_id=3XGPJOYMTCQCA5PJPKXBI4QZBRLFK67IRTLB434LEMWLNCZOMXHQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.556227130 (21h58m31.27879017s) [noise=true , address=104.36.82.7:4909 , piece_id=DVC47Y53A73WTZJ5CEAM6M5NTFWFNEXPRVZJV6RZSQLXGSIN7QCA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.589316551 (21h58m31.311879591s) [noise=true , address=45.91.116.4:4939 , piece_id=JVCWPOF2CLF2WBVBZZQSAZ3KP3XVPMYUXC7HG5QVICE6MC23VCNA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.595855452 (21h58m31.318418492s) [noise=true , address=45.91.116.9:4263 , piece_id=4MSCHEXYQET4LCKMS2DRXZUDLJEEGBJRDPPRD7FV5HIZYGTQE46Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.599202887 (21h58m31.321765927s) [noise=true , address=45.91.116.4:4938 , piece_id=EZOQH3VH4DHZQZNPQZJL3SQ3PD5EOHQMD3JRGOYZI2ETJHSE4W7A, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.623718644 (21h58m31.346281684s) [noise=true , address=104.36.82.8:4970 , piece_id=MW4ZOSF4XPKSX3GVQE44U6ZIKDWWRATC2YRHOLLSVOLBLVITR6MQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663640467 (21h58m31.386203507s) [noise=true , address=45.91.116.5:4297 , piece_id=MD2EEVFUOLH6WJEFLM7KFBLWUGLBGWDSOAAL3ON56DJMEMPAD37Q, seg_pos={0 0}, upload=21099]: Done uploading piece. err: <nil> Jul 25 09:34:21.663689460 (21h58m31.3862525s) [seg_pos={0 0}, upload=21099]: Segment reached optimal threshold of 13 pieces. Jul 25 09:34:21.663792598 (21h58m31.386355638s) [noise=true , address=104.36.82.15:4258 , piece_id=OBQ3LQJUCGAZCEEO4T76MXDEPVOUKACTKOO6YR35GCS6LJ7GSXEQ, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663848695 (21h58m31.386411725s) [noise=true , address=45.91.116.9:4262 , piece_id=YF6TLUROKJZYIV5ARC22MEZ34WRAJCEBGKP5VSLFXUOF53CZTOVA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663942966 (21h58m31.386506046s) [noise=true , address=104.36.82.15:4259 , piece_id=D64MTLWFIGPHK4OCHEF3WEY4I2DPPIGIG3UTJXVO5EC77TWV6TMA, seg_pos={0 0}, upload=21099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:21.663986699 (21h58m31.386549739s) [seg_pos={0 0}, upload=21099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 25 09:34:21.664102931 (21h58m31.386665961s) [upload=21099]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:21.664107882 (21h58m31.386670922s) [upload=21099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:21.949888312 (21h58m31.672451352s) [upload=21099]: Done uploading segments... Jul 25 09:34:21.949900605 (21h58m31.672463645s) [upload=21099]: Upload finished. err: <nil> Duration: 6.122367912s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................X..........X....XX......................] duration: 5.660202005s FAIL (2025-07-25 09:51:01 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 5.660228434s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.13652129s FAIL (2025-07-25 09:51:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.136544215s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [......................XX...X.........X....X.................] duration: 1.842091188s FAIL (2025-07-25 09:50:56 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.842114463s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.752343301s FAIL (2025-07-25 09:50:44 UTC) orbiter warning: uplink: encryption: context deadline exceeded Duration: 5.752368299s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 5.810903598s FAIL (2025-07-25 09:50:28 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971047854 (21h57m29.693610894s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971403364 (21h57m29.693966404s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971496193 (21h57m29.694059233s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971645788 (21h57m29.694208828s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.971861501 (21h57m29.694424541s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972111149 (21h57m29.694674189s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972179990 (21h57m29.69474303s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972413878 (21h57m29.694976918s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972672573 (21h57m29.695235613s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.972959813 (21h57m29.695522843s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973182669 (21h57m29.695745709s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973389165 (21h57m29.695952205s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.973681724 (21h57m29.696244764s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.974139269 (21h57m29.69670231s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.979076505 (21h57m29.701639545s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980483939 (21h57m29.703046979s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:19.980924963 (21h57m29.703488013s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Uploading piece... Jul 25 09:33:21.078720342 (21h57m30.801283382s) [upload=21087]: Next returned nil segment Jul 25 09:33:21.078738006 (21h57m30.801301046s) [upload=21087]: Waiting for error group managing segments... Jul 25 09:33:21.363916695 (21h57m31.086479735s) [noise=true , address=79.127.205.245:4231 , piece_id=JU7S3AZVWNUISTSFQJN6XXZFRPBL7GYJCO2M2MA7KY6IVSYMTHUA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.364254361 (21h57m31.086817401s) [noise=true , address=79.127.205.234:4606 , piece_id=EK6WNTDSNRXMLCTLZT7CHZQ7QNBFL5XSPUV4EU4KC4T6NVAR67WQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.390536257 (21h57m31.113099297s) [noise=true , address=79.127.205.235:4674 , piece_id=4ML475TWGWC2Y2RY3VHGRSFPWYRAQ2UFHKIKTORPU46TYBLP6R3A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.655019760 (21h57m31.3775828s) [noise=true , address=79.127.220.98:4405 , piece_id=TNK3B6RYBOYOOTDBV2KUBO4L6FC2PN6ONPUNVSLFRTFOVHCLVEKQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.857280952 (21h57m31.579843992s) [noise=true , address=77.90.31.13:4634 , piece_id=WVSIYUNHDNUTLYR5E4RQ2RIMMMATTP2CEVMIRB4QZQJ45PTCICFQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.862295736 (21h57m31.584858766s) [noise=true , address=156.146.43.225:4465 , piece_id=M2OJIWQPLJTDW2BV5MWN2P2BYUJ5DAQIRK6D2CIEC5Y72NQALOHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.866435047 (21h57m31.588998087s) [noise=true , address=77.90.31.17:4782 , piece_id=636XSXJ2USCIXE2FGVQ6OQKKYPS4TLNHQUS4D643TTJZB7WNRWWA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.869176071 (21h57m31.591739111s) [noise=true , address=45.91.116.14:4219 , piece_id=M5JEUIMM4DMFZQSGOCUN3EOOVXSZCNGGVBIXZLXE6H7DQRM2MB2A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.875093543 (21h57m31.597656583s) [noise=true , address=45.91.116.10:4091 , piece_id=FDCEZMOWGUIZ2EYSDEZGAOTAS47YSP4DTPGG4JUUWJ7CNGTXTV4Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.879395764 (21h57m31.601958804s) [noise=true , address=79.127.201.215:4355 , piece_id=26CYZFNL2JDZIERMZUQ6YKMQRWMS34TKG4LHSTUZ4KHJ5AM7X32A, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.882687593 (21h57m31.605250633s) [noise=true , address=45.91.116.4:4959 , piece_id=VNESVCXUAKIYTCW3YXW5Y47SGSDFK5LKUMED4WDXUDA5GLSCUBEA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.924284271 (21h57m31.646847311s) [noise=true , address=45.91.116.7:4036 , piece_id=P2HYAWWCORK77ASKFALQS5AMZIIUQ6JAAQFLQEQQJRCUAFDAVEOA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.927357140 (21h57m31.64992018s) [noise=true , address=79.127.219.47:4064 , piece_id=HXDSBIIAVBEI6UMD6GBTJYISEJ25PDZMN7IVBI42COHNU5Y3EVAA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969158079 (21h57m31.691721109s) [noise=true , address=104.36.82.12:4152 , piece_id=VS4VPALRH4VP2H7PNMSQG6M3RTOTBIAS5QT5C6ZVU5G4IGVBLBHA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: <nil> Jul 25 09:33:21.969181544 (21h57m31.691744584s) [seg_pos={0 0}, upload=21087]: Segment reached optimal threshold of 14 pieces. Jul 25 09:33:21.969288348 (21h57m31.691851388s) [noise=true , address=77.90.31.12:4619 , piece_id=4PJO2V6F26NXB4MU4TKI22UD6W2K3U3YY7CDKKVVYMJSXEK7ZPRQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969341660 (21h57m31.69190469s) [noise=true , address=77.90.31.15:4718 , piece_id=QEE223TXHFUKDE7S6G7MVEC3B34TF432LFH64KNKCPZJHNP2ZIDA, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P53pBJhZqd4Kn47H6UcmxkDSh11qKmUaMhKTk3bGBF4FbmgMC): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969385193 (21h57m31.691948233s) [noise=true , address=104.36.82.3:4797 , piece_id=32JD5L3GVJNF4LNJZNPDPAA3Y3JITLBAYNSAQFVUIW2JHALJRE5Q, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uC4tCyRnreNPqyXpsU6K8rPqShbaRHRgNX6amTf6aL7j23rQu): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969415982 (21h57m31.691979012s) [noise=true , address=77.90.31.16:4748 , piece_id=IEL4WPANVLJNSJNOH6VDS7EYZNXNO5YHC7PMKAYX7RSV7YKTFNEQ, seg_pos={0 0}, upload=21087]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k122BWfbKgBxSs8MCfNFkZYh2WfZDDHLktLQRiifkPr95G7SY): protocol: expected piece hash; context canceled; EOF Jul 25 09:33:21.969429789 (21h57m31.691992819s) [seg_pos={0 0}, upload=21087]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:33:21.969511695 (21h57m31.692074735s) [upload=21087]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:33:21.969516544 (21h57m31.692079575s) [upload=21087]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:22.271564791 (21h57m31.994127821s) [upload=21087]: Done uploading segments... Jul 25 09:33:22.271593316 (21h57m31.994156356s) [upload=21087]: Upload finished. err: <nil> Jul 25 09:35:12.402466382 (21h59m22.125029422s) [upload=21110]: Starting upload Jul 25 09:35:12.402529432 (21h59m22.125092472s) [upload=21110]: Done starting upload Jul 25 09:35:12.402603614 (21h59m22.125166654s) [upload=21110]: Uploading segments... Jul 25 09:35:12.402814256 (21h59m22.125377286s) [upload=21110]: Got next segment. Inline: false Jul 25 09:35:12.402839715 (21h59m22.125402755s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:12.402844614 (21h59m22.125407654s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:11.758724485 (22h0m21.481287515s) [upload=21110]: Next segment err: context deadline exceeded Jul 25 09:36:11.758860395 (22h0m21.481423435s) [upload=21110]: Done uploading segments... Jul 25 09:36:11.758890853 (22h0m21.481453893s) [upload=21110]: Upload finished. err: context deadline exceeded Duration: 5.810932192s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.X..................] duration: 1.985869686s FAIL (2025-07-25 09:50:21 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ... Jul 25 09:26:34.570036631 (21h50m44.292599671s) [upload=20980]: Upload finished. err: <nil> Jul 25 09:26:49.200928904 (21h50m58.923491944s) [upload=20986]: Starting upload Jul 25 09:26:49.200980633 (21h50m58.923543673s) [upload=20986]: Done starting upload Jul 25 09:26:49.201056208 (21h50m58.923619248s) [upload=20986]: Uploading segments... Jul 25 09:26:49.201090293 (21h50m58.923653333s) [upload=20986]: Got next segment. Inline: true Jul 25 09:26:49.201094100 (21h50m58.92365714s) [upload=20986]: Waiting for error group managing segments... Jul 25 09:26:49.201126071 (21h50m58.923689111s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:26:49.201130601 (21h50m58.923693641s) [upload=20986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:26:49.201133486 (21h50m58.923696526s) [upload=20986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:26:49.604840728 (21h50m59.327403768s) [upload=20986]: Done uploading segments... Jul 25 09:26:49.604861357 (21h50m59.327424398s) [upload=20986]: Upload finished. err: <nil> Jul 25 09:27:53.078396941 (21h52m2.800959981s) [upload=21002]: Starting upload Jul 25 09:27:53.078484157 (21h52m2.801047197s) [upload=21002]: Done starting upload Jul 25 09:27:53.078544112 (21h52m2.801107142s) [upload=21002]: Uploading segments... Jul 25 09:27:53.078584009 (21h52m2.801147039s) [upload=21002]: Got next segment. Inline: true Jul 25 09:27:53.078588026 (21h52m2.801151066s) [upload=21002]: Waiting for error group managing segments... Jul 25 09:27:53.078630116 (21h52m2.801193156s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:53.078634464 (21h52m2.801197504s) [upload=21002]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:53.078636810 (21h52m2.80119985s) [upload=21002]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:53.336646507 (21h52m3.059209587s) [upload=21002]: Done uploading segments... Jul 25 09:27:53.336667287 (21h52m3.059230337s) [upload=21002]: Upload finished. err: <nil> Jul 25 09:29:06.978452311 (21h53m16.701015351s) [upload=21021]: Starting upload Jul 25 09:29:06.978520191 (21h53m16.701083221s) [upload=21021]: Done starting upload Jul 25 09:29:06.978597849 (21h53m16.701160889s) [upload=21021]: Uploading segments... Jul 25 09:29:06.978698231 (21h53m16.701261281s) [upload=21021]: Got next segment. Inline: true Jul 25 09:29:06.978706287 (21h53m16.701269347s) [upload=21021]: Waiting for error group managing segments... Jul 25 09:29:06.978753848 (21h53m16.701316878s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:06.978758897 (21h53m16.701321937s) [upload=21021]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:06.978761131 (21h53m16.701324171s) [upload=21021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:07.240912794 (21h53m16.963475834s) [upload=21021]: Done uploading segments... Jul 25 09:29:07.240931219 (21h53m16.963494259s) [upload=21021]: Upload finished. err: <nil> Jul 25 09:30:11.152587399 (21h54m20.875150439s) [upload=21037]: Starting upload Jul 25 09:30:11.152643596 (21h54m20.875206636s) [upload=21037]: Done starting upload Jul 25 09:30:11.152684424 (21h54m20.875247464s) [upload=21037]: Uploading segments... Jul 25 09:30:11.152739640 (21h54m20.87530268s) [upload=21037]: Got next segment. Inline: true Jul 25 09:30:11.152744549 (21h54m20.875307589s) [upload=21037]: Waiting for error group managing segments... Jul 25 09:30:11.152785127 (21h54m20.875348167s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:11.152790467 (21h54m20.875353507s) [upload=21037]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:11.152793052 (21h54m20.875356092s) [upload=21037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:11.421977889 (21h54m21.144540919s) [upload=21037]: Done uploading segments... Jul 25 09:30:11.421995103 (21h54m21.144558143s) [upload=21037]: Upload finished. err: <nil> Jul 25 09:31:21.388209097 (21h55m31.110772137s) [upload=21053]: Starting upload Jul 25 09:31:21.388300943 (21h55m31.110863983s) [upload=21053]: Done starting upload Jul 25 09:31:21.388424629 (21h55m31.110987669s) [upload=21053]: Uploading segments... Jul 25 09:31:21.388526104 (21h55m31.111089144s) [upload=21053]: Got next segment. Inline: true Jul 25 09:31:21.388530682 (21h55m31.111093722s) [upload=21053]: Waiting for error group managing segments... Jul 25 09:31:21.388591739 (21h55m31.111154779s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:21.388600687 (21h55m31.111163727s) [upload=21053]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:21.388603402 (21h55m31.111166442s) [upload=21053]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:21.651857102 (21h55m31.374420132s) [upload=21053]: Done uploading segments... Jul 25 09:31:21.651874595 (21h55m31.374437625s) [upload=21053]: Upload finished. err: <nil> Jul 25 09:32:08.831277463 (21h56m18.553840503s) [upload=21065]: Starting upload Jul 25 09:32:08.831391121 (21h56m18.553954171s) [upload=21065]: Done starting upload Jul 25 09:32:08.831498536 (21h56m18.554061576s) [upload=21065]: Uploading segments... Jul 25 09:32:08.831577387 (21h56m18.554140437s) [upload=21065]: Got next segment. Inline: true Jul 25 09:32:08.831583599 (21h56m18.554146639s) [upload=21065]: Waiting for error group managing segments... Jul 25 09:32:08.831632082 (21h56m18.554195122s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:08.831638153 (21h56m18.554201193s) [upload=21065]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:08.831658842 (21h56m18.554221882s) [upload=21065]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:09.099387434 (21h56m18.821950484s) [upload=21065]: Done uploading segments... Jul 25 09:32:09.099442660 (21h56m18.8220057s) [upload=21065]: Upload finished. err: <nil> Jul 25 09:33:13.963999397 (21h57m23.686562427s) [upload=21085]: Starting upload Jul 25 09:33:13.964075112 (21h57m23.686638152s) [upload=21085]: Done starting upload Jul 25 09:33:13.964129967 (21h57m23.686693007s) [upload=21085]: Uploading segments... Jul 25 09:33:13.964160586 (21h57m23.686723626s) [upload=21085]: Got next segment. Inline: true Jul 25 09:33:13.964164894 (21h57m23.686727934s) [upload=21085]: Waiting for error group managing segments... Jul 25 09:33:13.964202505 (21h57m23.686765545s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:13.964207115 (21h57m23.686770155s) [upload=21085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:13.964209650 (21h57m23.68677269s) [upload=21085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:14.232920338 (21h57m23.955483378s) [upload=21085]: Done uploading segments... Jul 25 09:33:14.232938894 (21h57m23.955501934s) [upload=21085]: Upload finished. err: <nil> Jul 25 09:33:43.505781098 (21h57m53.228344148s) [upload=21092]: Starting upload Jul 25 09:33:43.505886630 (21h57m53.22844967s) [upload=21092]: Done starting upload Jul 25 09:33:43.505958236 (21h57m53.228521276s) [upload=21092]: Uploading segments... Jul 25 09:33:43.506013322 (21h57m53.228576362s) [upload=21092]: Got next segment. Inline: true Jul 25 09:33:43.506018011 (21h57m53.228581051s) [upload=21092]: Waiting for error group managing segments... Jul 25 09:33:43.506062997 (21h57m53.228626037s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:43.506068237 (21h57m53.228631277s) [upload=21092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:43.506071313 (21h57m53.228634353s) [upload=21092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:43.974697177 (21h57m53.697260217s) [upload=21092]: Done uploading segments... Jul 25 09:33:43.974715503 (21h57m53.697278543s) [upload=21092]: Upload finished. err: <nil> Jul 25 09:35:21.634207561 (21h59m31.356770601s) [upload=21112]: Starting upload Jul 25 09:35:21.634263188 (21h59m31.356826228s) [upload=21112]: Done starting upload Jul 25 09:35:21.634309175 (21h59m31.356872215s) [upload=21112]: Uploading segments... Jul 25 09:35:21.634341738 (21h59m31.356904778s) [upload=21112]: Got next segment. Inline: true Jul 25 09:35:21.634345235 (21h59m31.356908275s) [upload=21112]: Waiting for error group managing segments... Jul 25 09:35:21.634382045 (21h59m31.356945075s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:21.634386453 (21h59m31.356949493s) [upload=21112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:35:21.634388888 (21h59m31.356951918s) [upload=21112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.912198648 (21h59m31.634761688s) [upload=21112]: Done uploading segments... Jul 25 09:35:21.912214678 (21h59m31.634777719s) [upload=21112]: Upload finished. err: <nil> Jul 25 09:36:16.777669823 (22h0m26.500232863s) [upload=21123]: Starting upload Jul 25 09:36:16.777732162 (22h0m26.500295202s) [upload=21123]: Done starting upload Jul 25 09:36:16.777828126 (22h0m26.500391166s) [upload=21123]: Uploading segments... Jul 25 09:36:16.777862512 (22h0m26.500425542s) [upload=21123]: Got next segment. Inline: true Jul 25 09:36:16.777866379 (22h0m26.500429419s) [upload=21123]: Waiting for error group managing segments... Jul 25 09:36:16.777902208 (22h0m26.500465248s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:36:16.777908921 (22h0m26.500471961s) [upload=21123]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:36:16.777911355 (22h0m26.500474395s) [upload=21123]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:36:21.722625410 (22h0m31.44518845s) [upload=21123]: Done uploading segments... Jul 25 09:36:21.722650267 (22h0m31.445213318s) [upload=21123]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.985902728s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.927631646s FAIL (2025-07-25 09:50:16 UTC) orbiter warning: uplink: metaclient: context deadline exceeded Duration: 1.927656424s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.892298792s FAIL (2025-07-25 09:50:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.892316085s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X..........X....XX.....................] duration: 5.779869596s FAIL (2025-07-25 09:50:09 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 5.779901006s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)