Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X................................X] duration:
4.293370598s
FAIL (2025-08-08 17:59:01 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
Aug 8 17:58:57.687589430 (32m52.118669561s) [upload=588]: Starting upload
Aug 8 17:58:57.687627203 (32m52.118707324s) [upload=588]: Done starting upload
Aug 8 17:58:57.687821034 (32m52.118901165s) [upload=588]: Uploading segments...
Aug 8 17:58:57.687860369 (32m52.1189405s) [upload=588]: Got next segment. Inline: false
Aug 8 17:58:57.687886398 (32m52.11896653s) [upload=588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 17:58:57.687890406 (32m52.118970537s) [upload=588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 17:58:58.147562561 (32m52.578642682s) [seg_pos={0 0}, upload=588]: Begin upload segment...
Aug 8 17:58:58.147605704 (32m52.578685825s) [seg_pos={0 0}, upload=588]: Done begin upload segment.
Aug 8 17:58:58.147625301 (32m52.578705422s) [noise=true , address=79.127.219.51:4181 , piece_id=7GOUGLRJVZKW45LDJXAUJT5A5TQPPOTXXA3OW5BB2WZIAHIFFKKA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.147979318 (32m52.579059449s) [noise=true , address=104.36.82.35:4594 , piece_id=6RX25NVO6OMF6N2G5DK5VAZSULO47IGQVUROL7367DZJXVM7JTUA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.148253864 (32m52.579333995s) [noise=true , address=104.36.82.44:4877 , piece_id=Y4NTQVMER2Z4HFCFCPGFMLU3FNCVAHP6FBYZX22G6T6VBYMYFWXQ, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.148899449 (32m52.57997958s) [noise=true , address=104.36.82.2:4761 , piece_id=6SRL2DT565OCQ43KVC7AX6WC63HURQK226IYIS3HF545KHOZCNLQ, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.149130150 (32m52.580210271s) [noise=true , address=104.36.82.61:4373 , piece_id=ZXUOSUCOMB3CEJX5AHT5ZAOW2ZKOEWGN7JB3YF3NZQ3FGOE4BEAQ, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.149356184 (32m52.580436315s) [noise=true , address=104.36.82.6:4878 , piece_id=UXFCCM5YNLOHLETFQZLNJX3XRHUNMIF44EUGRKPOY4DO2VSPT4BA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.149617223 (32m52.580697354s) [noise=true , address=104.36.82.16:4310 , piece_id=H23ERP7EDZQDABZXZ7L7MYJKVHPK4AU57VWUVNLSV6FHS36UUTXQ, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.149867892 (32m52.580948023s) [noise=true , address=104.36.82.47:4534 , piece_id=BJKFSI2OIJGZYMY76LGCNBQJGWNTVS7N6RVFMYIRDBMWIJCWTJJA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.150167736 (32m52.581247857s) [noise=true , address=79.127.219.51:4198 , piece_id=MWXX3OLOMRZSCVUTBIAFIOJIVWTDHQC6UPQTJKDHTUA6RPVCCTEA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.150400712 (32m52.581480843s) [noise=true , address=104.36.82.23:4458 , piece_id=CKKNPTVUHUB6X4INP33HBMPYDJ5GWI4OQRRRZCGMLILLROVLFRUA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.150698872 (32m52.581779003s) [noise=true , address=104.36.82.43:4853 , piece_id=PDRJVWFVOEPJIBQBDU3HKIZIJBUHHHSDRANZI2HIBBFCPYFQLAIA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.150942037 (32m52.582022168s) [noise=true , address=104.36.82.59:4327 , piece_id=JN2ISE4TGMJJV545UOACR55PPGJ37Q6BGKMTYJD7NJPIFXKWXTHA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.151172879 (32m52.58225301s) [noise=true , address=104.36.82.36:4648 , piece_id=4ORRZDPRIDZHVOK6GFRRTNWIQ6KPRGOXVTTQCITETEMIWBXETMIA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.151389795 (32m52.582469916s) [noise=true , address=104.36.82.19:4406 , piece_id=5Q37QRWCVPXARQFMWGXFEAAEESR2VZO6LW7QFXAHPJNM7BBUTMEA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.151635233 (32m52.582715354s) [noise=true , address=104.36.82.34:4580 , piece_id=YR34U5ODNLREKEAB7FCHBJ2TSPCJEGQDMQBXKPL5CJUJC52ZI6QQ, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.151869893 (32m52.582950024s) [noise=true , address=45.91.116.9:4275 , piece_id=4VMJ753VDOF46C4O6PJJHGFDP5PDPN3ON457HPEL7AY3SENIBTRA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.152072881 (32m52.583153012s) [noise=true , address=45.91.116.13:4185 , piece_id=JVNOONDOW3BGCNBFPM2553OTUSVVEWTPZVFTWFAOLJQOZBNDHJHA, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:58.152285267 (32m52.583365398s) [noise=true , address=104.36.82.12:4165 , piece_id=YENUQ3Z5EFXJ5RFHVSXK73OCGO4IUIVHWN4AZIDXL6L3D64R7GAQ, seg_pos={0 0}, upload=588]: Uploading piece...
Aug 8 17:58:59.654634717 (32m54.085714858s) [upload=588]: Next returned nil segment
Aug 8 17:58:59.654651438 (32m54.085731569s) [upload=588]: Waiting for error group managing segments...
Aug 8 17:59:00.067695971 (32m54.498776102s) [noise=true , address=104.36.82.43:4853 , piece_id=PDRJVWFVOEPJIBQBDU3HKIZIJBUHHHSDRANZI2HIBBFCPYFQLAIA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.105330478 (32m54.536410609s) [noise=true , address=79.127.219.51:4198 , piece_id=MWXX3OLOMRZSCVUTBIAFIOJIVWTDHQC6UPQTJKDHTUA6RPVCCTEA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.146519144 (32m54.577599275s) [noise=true , address=104.36.82.23:4458 , piece_id=CKKNPTVUHUB6X4INP33HBMPYDJ5GWI4OQRRRZCGMLILLROVLFRUA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.149019347 (32m54.580099468s) [noise=true , address=104.36.82.2:4761 , piece_id=6SRL2DT565OCQ43KVC7AX6WC63HURQK226IYIS3HF545KHOZCNLQ, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.154663226 (32m54.585743357s) [noise=true , address=104.36.82.6:4878 , piece_id=UXFCCM5YNLOHLETFQZLNJX3XRHUNMIF44EUGRKPOY4DO2VSPT4BA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.159408265 (32m54.590488386s) [noise=true , address=104.36.82.12:4165 , piece_id=YENUQ3Z5EFXJ5RFHVSXK73OCGO4IUIVHWN4AZIDXL6L3D64R7GAQ, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.168359110 (32m54.599439241s) [noise=true , address=104.36.82.44:4877 , piece_id=Y4NTQVMER2Z4HFCFCPGFMLU3FNCVAHP6FBYZX22G6T6VBYMYFWXQ, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.186862202 (32m54.617942323s) [noise=true , address=104.36.82.34:4580 , piece_id=YR34U5ODNLREKEAB7FCHBJ2TSPCJEGQDMQBXKPL5CJUJC52ZI6QQ, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.206628962 (32m54.637709083s) [noise=true , address=104.36.82.16:4310 , piece_id=H23ERP7EDZQDABZXZ7L7MYJKVHPK4AU57VWUVNLSV6FHS36UUTXQ, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.207036983 (32m54.638117194s) [noise=true , address=45.91.116.9:4275 , piece_id=4VMJ753VDOF46C4O6PJJHGFDP5PDPN3ON457HPEL7AY3SENIBTRA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.228579751 (32m54.659659882s) [noise=true , address=104.36.82.36:4648 , piece_id=4ORRZDPRIDZHVOK6GFRRTNWIQ6KPRGOXVTTQCITETEMIWBXETMIA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.229354534 (32m54.660434665s) [noise=true , address=104.36.82.19:4406 , piece_id=5Q37QRWCVPXARQFMWGXFEAAEESR2VZO6LW7QFXAHPJNM7BBUTMEA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.234562829 (32m54.66564296s) [noise=true , address=45.91.116.13:4185 , piece_id=JVNOONDOW3BGCNBFPM2553OTUSVVEWTPZVFTWFAOLJQOZBNDHJHA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.243158605 (32m54.674238736s) [noise=true , address=104.36.82.59:4327 , piece_id=JN2ISE4TGMJJV545UOACR55PPGJ37Q6BGKMTYJD7NJPIFXKWXTHA, seg_pos={0 0}, upload=588]: Done uploading piece. err: <nil>
Aug 8 17:59:00.243184965 (32m54.674265096s) [seg_pos={0 0}, upload=588]: Segment reached optimal threshold of 14 pieces.
Aug 8 17:59:00.243289776 (32m54.674369907s) [noise=true , address=104.36.82.47:4534 , piece_id=BJKFSI2OIJGZYMY76LGCNBQJGWNTVS7N6RVFMYIRDBMWIJCWTJJA, seg_pos={0 0}, upload=588]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr): protocol: expected piece hash; context canceled; EOF
Aug 8 17:59:00.243360421 (32m54.674440542s) [noise=true , address=79.127.219.51:4181 , piece_id=7GOUGLRJVZKW45LDJXAUJT5A5TQPPOTXXA3OW5BB2WZIAHIFFKKA, seg_pos={0 0}, upload=588]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KvEKktxRuXnEgc6rcJj8rotGamhqt3pcEHSiBkXMkyRC2uCRB): protocol: expected piece hash; context canceled; EOF
Aug 8 17:59:00.243410096 (32m54.674490227s) [noise=true , address=104.36.82.35:4594 , piece_id=6RX25NVO6OMF6N2G5DK5VAZSULO47IGQVUROL7367DZJXVM7JTUA, seg_pos={0 0}, upload=588]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf): protocol: expected piece hash; context canceled; EOF
Aug 8 17:59:00.243438971 (32m54.674519092s) [noise=true , address=104.36.82.61:4373 , piece_id=ZXUOSUCOMB3CEJX5AHT5ZAOW2ZKOEWGN7JB3YF3NZQ3FGOE4BEAQ, seg_pos={0 0}, upload=588]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AQGSkJuNJDxoeqhChUenWxHwF5yK41UESCF63FQz3XvabXXfz): protocol: expected piece hash; context canceled; EOF
Aug 8 17:59:00.243485631 (32m54.674565762s) [seg_pos={0 0}, upload=588]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 17:59:00.243578738 (32m54.674658869s) [upload=588]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 17:59:00.243582997 (32m54.674663128s) [upload=588]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 17:59:00.666727024 (32m55.097807165s) [upload=588]: Done uploading segments...
Aug 8 17:59:00.666749676 (32m55.097829807s) [upload=588]: Upload finished. err: <nil>
Duration: 4.293391987s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.016637395s
FAIL (2025-08-08 17:58:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.016652023s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................................X..........X] duration:
1.43011378s
FAIL (2025-08-08 17:58:38 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 8 17:58:37.957200338 (32m32.388280469s) [upload=582]: Starting upload Aug 8 17:58:37.957297144 (32m32.388377275s) [upload=582]: Done starting upload Aug 8 17:58:37.957394770 (32m32.388474901s) [upload=582]: Uploading segments... Aug 8 17:58:37.957482327 (32m32.388562459s) [upload=582]: Got next segment. Inline: true Aug 8 17:58:37.957488490 (32m32.388568631s) [upload=582]: Waiting for error group managing segments... Aug 8 17:58:37.957554776 (32m32.388634907s) [upload=582]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:58:37.957561689 (32m32.38864182s) [upload=582]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:58:37.957565306 (32m32.388645427s) [upload=582]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:58:38.276535375 (32m32.707615516s) [upload=582]: Done uploading segments... Aug 8 17:58:38.276578197 (32m32.707658339s) [upload=582]: Upload finished. err: <nil>
Duration: 1.430134001s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.936124899s
FAIL (2025-08-08 17:58:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.936148605s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.895504112s
FAIL (2025-08-08 17:58:05 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.895526646s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.012519001s
FAIL (2025-08-08 17:57:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.012535433s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
6.050303277s
FAIL (2025-08-08 17:57:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.05032639s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.072671817s
FAIL (2025-08-08 17:57:21 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 2.07269478s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.965860979s
FAIL (2025-08-08 17:57:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.965875857s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
6.017995095s
FAIL (2025-08-08 17:57:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.018020994s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.839897318s
FAIL (2025-08-08 17:56:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.839917376s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.368962175s
FAIL (2025-08-08 17:56:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.369024764s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.176043171s
FAIL (2025-08-08 17:56:13 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.176057879s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.841894243s
FAIL (2025-08-08 17:55:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.841916697s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.887083258s
FAIL (2025-08-08 17:55:31 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.887107395s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.861508096s
FAIL (2025-08-08 17:55:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.861528825s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.849698058s
FAIL (2025-08-08 17:55:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.849717736s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.167665874s
FAIL (2025-08-08 17:54:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.167685011s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
6.01111996s
FAIL (2025-08-08 17:54:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.011146491s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.00525982s
FAIL (2025-08-08 17:54:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.00527524s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.735982014s
FAIL (2025-08-08 17:54:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.736038872s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.963910572s
FAIL (2025-08-08 17:54:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.96393547s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
6.049852059s
FAIL (2025-08-08 17:53:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.04987304s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
6.120579892s
FAIL (2025-08-08 17:53:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.120600851s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.776822488s
FAIL (2025-08-08 17:53:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.776844792s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.877577362s
FAIL (2025-08-08 17:53:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.877599084s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.895704539s
FAIL (2025-08-08 17:52:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.895722864s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
6.008555156s
FAIL (2025-08-08 17:52:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.008574252s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.569231825s
FAIL (2025-08-08 17:52:24 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.569256263s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.961677899s
FAIL (2025-08-08 17:52:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.961706845s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.021285077s
FAIL (2025-08-08 17:51:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 2.021322089s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
6.417556519s
FAIL (2025-08-08 17:51:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.41758368s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001297698s
FAIL (2025-08-08 17:51:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001318678s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
6.028705078s
FAIL (2025-08-08 17:51:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.028730016s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.877642525s
FAIL (2025-08-08 17:51:13 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.877665258s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.869576648s
FAIL (2025-08-08 17:50:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.869596624s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1.641085629s
FAIL (2025-08-08 17:50:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 84cf772ada8114705c1b2a6f7bc7364b1b0532350c5e582b80c3c54bc36ea324 != before 85637bd7abd0f00bf3e65d4c46ec31ffbfb4ccfefe2941a711626da6a5d91067 ======LOGS====== Aug 8 17:50:27.049701250 (24m21.480781371s) [upload=433]: Starting upload Aug 8 17:50:27.049761194 (24m21.480841325s) [upload=433]: Done starting upload Aug 8 17:50:27.049834665 (24m21.480914806s) [upload=433]: Uploading segments... Aug 8 17:50:27.049871345 (24m21.480951476s) [upload=433]: Got next segment. Inline: true Aug 8 17:50:27.049875474 (24m21.480955605s) [upload=433]: Waiting for error group managing segments... Aug 8 17:50:27.049924827 (24m21.481004948s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:50:27.049928966 (24m21.481009097s) [upload=433]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 17:50:27.049931481 (24m21.481011612s) [upload=433]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 17:50:27.359476715 (24m21.790556846s) [upload=433]: Done uploading segments... Aug 8 17:50:27.359498637 (24m21.790578828s) [upload=433]: Upload finished. err: <nil>
Duration: 1.641112651s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.997840563s
FAIL (2025-08-08 17:50:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.997863045s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.985286825s
FAIL (2025-08-08 17:49:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.985315832s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.762299821s
FAIL (2025-08-08 17:48:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.762319788s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.895408199s
FAIL (2025-08-08 17:47:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.895428468s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.795959035s
FAIL (2025-08-08 17:46:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.795986957s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.702452514s
FAIL (2025-08-08 17:45:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.70248157s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.925365677s
FAIL (2025-08-08 17:44:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.925393159s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.768044184s
FAIL (2025-08-08 17:43:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.768072948s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.0341427s
FAIL (2025-08-08 17:42:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.034180103s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.890438923s
FAIL (2025-08-08 17:41:47 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.890462218s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.015128884s
FAIL (2025-08-08 17:41:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.015155687s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.996324946s
FAIL (2025-08-08 17:40:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.996350516s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.395122278s
FAIL (2025-08-08 17:39:38 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 17:34:04.423991702 (7m58.855071823s) [upload=143]: Starting upload Aug 8 17:34:04.424039194 (7m58.855119325s) [upload=143]: Done starting upload Aug 8 17:34:04.424125088 (7m58.855205219s) [upload=143]: Uploading segments... Aug 8 17:34:04.424216153 (7m58.855296284s) [upload=143]: Got next segment. Inline: false Aug 8 17:34:04.424248705 (7m58.855328836s) [upload=143]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 17:34:04.424253183 (7m58.855333314s) [upload=143]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 8 17:34:04.636703898 (7m59.067784019s) [upload=143]: Next segment err: metaclient: Too Many Requests Aug 8 17:34:04.636724957 (7m59.067805088s) [upload=143]: Done uploading segments... Aug 8 17:34:04.636740257 (7m59.067820388s) [upload=143]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.395192802s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|