Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
10 out of 60 is failed (threshold=3) [...............................X..........X.XX......XX.XX.XX] duration:
30.002024866s
FAIL (2025-07-29 18:06:20 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 30.002046057s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.001920461s
FAIL (2025-07-29 18:05:58 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
IAYHKNHQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:56648->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:56648->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56648->45.91.116.11:4118: use of closed network connection
Jul 29 18:04:41.043389960 (25h18m53.519328425s) [seg_pos={0 0}, upload=24231]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:04:41.043545126 (25h18m53.519483591s) [upload=24231]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:04:41.043550778 (25h18m53.519489243s) [upload=24231]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:04:41.465788927 (25h18m53.941727392s) [upload=24231]: Done uploading segments...
Jul 29 18:04:41.465842159 (25h18m53.941780624s) [upload=24231]: Upload finished. err: <nil>
Jul 29 18:05:28.986432727 (25h19m41.462371192s) [upload=24242]: Starting upload
Jul 29 18:05:28.986491639 (25h19m41.462430094s) [upload=24242]: Done starting upload
Jul 29 18:05:28.986542938 (25h19m41.462481403s) [upload=24242]: Uploading segments...
Jul 29 18:05:28.986757388 (25h19m41.462695853s) [upload=24242]: Got next segment. Inline: false
Jul 29 18:05:28.986783709 (25h19m41.462722174s) [upload=24242]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 18:05:28.986788207 (25h19m41.462726672s) [upload=24242]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 18:05:29.249497366 (25h19m41.725435821s) [seg_pos={0 0}, upload=24242]: Begin upload segment...
Jul 29 18:05:29.249539306 (25h19m41.725477771s) [seg_pos={0 0}, upload=24242]: Done begin upload segment.
Jul 29 18:05:29.249564123 (25h19m41.725502588s) [noise=true , address=77.90.31.13:4627 , piece_id=KEOGIW5FWFCQKECJBNJ6O5DLJSTFBQOAMQZGNOSGTWJ4KDRH3UOA, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.249604131 (25h19m41.725542586s) [noise=true , address=77.90.31.16:4733 , piece_id=BBSO4JNO6QS7LBDGP7G7C675IQBBDJTXXNG3QS25G2FKFWDEZFQA, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.249941085 (25h19m41.72587955s) [noise=true , address=45.91.116.12:4225 , piece_id=VMGZUD5SOKDESZSQK3BAMHJG27GOAEFY2BOMRAPJ3NVKXTUUPDLA, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.249948940 (25h19m41.725887395s) [noise=true , address=77.90.31.11:4553 , piece_id=FVJHS7V5GKQUR4DEJXKEMHBPJM6MYAWFPDC7JCQXJEE5KFLXJC3Q, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.250202085 (25h19m41.72614055s) [noise=true , address=45.91.116.11:4118 , piece_id=GK4JF3RXPI4XWWYSDEODW2RERTHWBP6ENPTLALTB6WNTVFMDRVTQ, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.250214188 (25h19m41.726152643s) [noise=true , address=45.91.116.8:4047 , piece_id=5TM453FATCY2NHMTESTOAR4ZYOBD3WNAKYACRMMQ34EWJZUEZPSQ, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.250453075 (25h19m41.72639154s) [noise=true , address=104.36.82.9:4007 , piece_id=3ZBOLLCN6A7YQ2UIY3OU5NUGOUGVSBANC6SY2MYIK3VADF3QOLVQ, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.250466360 (25h19m41.726404825s) [noise=true , address=104.36.82.8:4969 , piece_id=6RSYLRP2ME6OKBDANNBD77TZEHWABCXCI6LVXD6KDMF46AAOWQLQ, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.250753690 (25h19m41.726692155s) [noise=true , address=77.90.31.18:4806 , piece_id=SNX5YHNAOBZWR32HKHWCV4L3HDK6HOTF7SOU7WG7DSOHTZC77D7A, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.250822171 (25h19m41.726760636s) [noise=true , address=104.36.82.15:4257 , piece_id=JT4VHXLRZ3BMRY7NWW4NU6QTDV5ZISNPW3POACOKLXKTZAKGOQ2Q, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.251006444 (25h19m41.726944909s) [noise=true , address=77.90.31.17:4772 , piece_id=TPUSDMZCAEDA7K5SMKR3NM7F3SDFXRXYTPYYAXAKZMF4I4APDHHA, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.251038525 (25h19m41.72697699s) [noise=true , address=77.90.31.14:4663 , piece_id=ISXY5DWAFL6JERL3OLJF6ECHONXJYIYJFB7FCIOJXX4EV6KF2C3Q, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.251204933 (25h19m41.727143388s) [noise=true , address=104.36.82.2:4732 , piece_id=YAXB3EKNAUJ44IPWP737SGQI6W7B6O623TGRTCTAUHXSZRZJLS4A, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.251245131 (25h19m41.727183596s) [noise=true , address=45.91.116.5:4299 , piece_id=VFNHG4NHNZTRAZEHLO6H5KJU5C3QDZFLQVKLZPG27RPYOPRAJW4A, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.251499737 (25h19m41.727438202s) [noise=true , address=104.36.82.13:4187 , piece_id=7HZAS3UP55FSLSDDSD456SQ75QQRFTYBTCATCSX4L5O6VJY4I2HA, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:29.251833907 (25h19m41.727772372s) [noise=true , address=77.90.31.10:4520 , piece_id=AUZ2VUSENXBWYSSS6EW2ZD5P5FALKQTBMERBEEPNJWARS5VBLMCA, seg_pos={0 0}, upload=24242]: Uploading piece...
Jul 29 18:05:31.253074765 (25h19m43.72901323s) [upload=24242]: Next returned nil segment
Jul 29 18:05:31.253096656 (25h19m43.729035121s) [upload=24242]: Waiting for error group managing segments...
Jul 29 18:05:31.636448074 (25h19m44.112386539s) [noise=true , address=45.91.116.12:4225 , piece_id=VMGZUD5SOKDESZSQK3BAMHJG27GOAEFY2BOMRAPJ3NVKXTUUPDLA, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:31.799471240 (25h19m44.275409705s) [noise=true , address=77.90.31.13:4627 , piece_id=KEOGIW5FWFCQKECJBNJ6O5DLJSTFBQOAMQZGNOSGTWJ4KDRH3UOA, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:31.890868011 (25h19m44.366806486s) [noise=true , address=45.91.116.8:4047 , piece_id=5TM453FATCY2NHMTESTOAR4ZYOBD3WNAKYACRMMQ34EWJZUEZPSQ, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:31.938774893 (25h19m44.414713348s) [noise=true , address=104.36.82.2:4732 , piece_id=YAXB3EKNAUJ44IPWP737SGQI6W7B6O623TGRTCTAUHXSZRZJLS4A, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:31.958293248 (25h19m44.434231703s) [noise=true , address=104.36.82.15:4257 , piece_id=JT4VHXLRZ3BMRY7NWW4NU6QTDV5ZISNPW3POACOKLXKTZAKGOQ2Q, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:31.973175653 (25h19m44.449114118s) [noise=true , address=45.91.116.5:4299 , piece_id=VFNHG4NHNZTRAZEHLO6H5KJU5C3QDZFLQVKLZPG27RPYOPRAJW4A, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.032954099 (25h19m44.508892564s) [noise=true , address=77.90.31.10:4520 , piece_id=AUZ2VUSENXBWYSSS6EW2ZD5P5FALKQTBMERBEEPNJWARS5VBLMCA, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.222196351 (25h19m44.698134816s) [noise=true , address=104.36.82.13:4187 , piece_id=7HZAS3UP55FSLSDDSD456SQ75QQRFTYBTCATCSX4L5O6VJY4I2HA, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.257729378 (25h19m44.733667843s) [noise=true , address=77.90.31.16:4733 , piece_id=BBSO4JNO6QS7LBDGP7G7C675IQBBDJTXXNG3QS25G2FKFWDEZFQA, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.259243835 (25h19m44.7351823s) [noise=true , address=45.91.116.11:4118 , piece_id=GK4JF3RXPI4XWWYSDEODW2RERTHWBP6ENPTLALTB6WNTVFMDRVTQ, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.431663290 (25h19m44.907601755s) [noise=true , address=77.90.31.14:4663 , piece_id=ISXY5DWAFL6JERL3OLJF6ECHONXJYIYJFB7FCIOJXX4EV6KF2C3Q, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.452746949 (25h19m44.928685414s) [noise=true , address=77.90.31.17:4772 , piece_id=TPUSDMZCAEDA7K5SMKR3NM7F3SDFXRXYTPYYAXAKZMF4I4APDHHA, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.502663977 (25h19m44.978602442s) [noise=true , address=104.36.82.8:4969 , piece_id=6RSYLRP2ME6OKBDANNBD77TZEHWABCXCI6LVXD6KDMF46AAOWQLQ, seg_pos={0 0}, upload=24242]: Done uploading piece. err: <nil>
Jul 29 18:05:32.502693494 (25h19m44.978631959s) [seg_pos={0 0}, upload=24242]: Segment reached optimal threshold of 13 pieces.
Jul 29 18:05:32.502820617 (25h19m44.978759082s) [noise=true , address=104.36.82.9:4007 , piece_id=3ZBOLLCN6A7YQ2UIY3OU5NUGOUGVSBANC6SY2MYIK3VADF3QOLVQ, seg_pos={0 0}, upload=24242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Jul 29 18:05:32.502839684 (25h19m44.978778149s) [noise=true , address=77.90.31.11:4553 , piece_id=FVJHS7V5GKQUR4DEJXKEMHBPJM6MYAWFPDC7JCQXJEE5KFLXJC3Q, seg_pos={0 0}, upload=24242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 29 18:05:32.502895421 (25h19m44.978833896s) [noise=true , address=77.90.31.18:4806 , piece_id=SNX5YHNAOBZWR32HKHWCV4L3HDK6HOTF7SOU7WG7DSOHTZC77D7A, seg_pos={0 0}, upload=24242]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 29 18:05:32.502914306 (25h19m44.978852781s) [seg_pos={0 0}, upload=24242]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:05:32.503010881 (25h19m44.978949346s) [upload=24242]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:05:32.503016191 (25h19m44.978954656s) [upload=24242]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:05:32.972533130 (25h19m45.448471595s) [upload=24242]: Done uploading segments...
Jul 29 18:05:32.972551957 (25h19m45.448490412s) [upload=24242]: Upload finished. err: <nil>
Duration: 30.001943685s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...X...X.................................X..................] duration:
1.92425508s
FAIL (2025-07-29 18:05:55 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.92430171s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
9 out of 60 is failed (threshold=3) [................................X..........X.XX......XX.XX.X] duration:
30.002246295s
FAIL (2025-07-29 18:05:26 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 30.002267075s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.00244689s
FAIL (2025-07-29 18:05:07 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4876: use of closed network connection; write tcp 188.166.225.170:50358->104.36.82.6:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50358->104.36.82.6:4876: use of closed network connection
Jul 29 18:03:36.280903070 (25h17m48.756841535s) [seg_pos={0 0}, upload=24215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:03:36.280964928 (25h17m48.756903393s) [upload=24215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:03:36.280968505 (25h17m48.75690697s) [upload=24215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:03:36.580401990 (25h17m49.056340455s) [upload=24215]: Done uploading segments...
Jul 29 18:03:36.580420275 (25h17m49.05635874s) [upload=24215]: Upload finished. err: <nil>
Jul 29 18:04:37.738415110 (25h18m50.214353565s) [upload=24231]: Starting upload
Jul 29 18:04:37.738467721 (25h18m50.214406176s) [upload=24231]: Done starting upload
Jul 29 18:04:37.738521855 (25h18m50.21446032s) [upload=24231]: Uploading segments...
Jul 29 18:04:37.738659808 (25h18m50.214598274s) [upload=24231]: Got next segment. Inline: false
Jul 29 18:04:37.738684746 (25h18m50.214623211s) [upload=24231]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 18:04:37.738688914 (25h18m50.214627379s) [upload=24231]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 18:04:37.974714369 (25h18m50.450652834s) [seg_pos={0 0}, upload=24231]: Begin upload segment...
Jul 29 18:04:37.974773131 (25h18m50.450711596s) [seg_pos={0 0}, upload=24231]: Done begin upload segment.
Jul 29 18:04:37.974794843 (25h18m50.450733308s) [noise=true , address=45.91.116.10:4083 , piece_id=NO77ZLIGYMBJOW4XS2EW2VVWAJJBCZQV7JT66QGNPF4PVUMPAJ7A, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.974838666 (25h18m50.450777131s) [noise=true , address=104.36.82.16:4295 , piece_id=CO4OE575M6EQXZP2KBPFLP57SQJMKGWKVT6E2HSS4GVACW75BECA, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.975210127 (25h18m50.451148592s) [noise=true , address=45.91.116.11:4118 , piece_id=GCJPBH7YYLU5V4KIMLMJ6GH67D5D43OIKRQX3YGMQWAKIAYHKNHQ, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.975325448 (25h18m50.451263913s) [noise=true , address=45.91.116.13:4154 , piece_id=YTYBO3FGNXYRCZLSNAKTUUT6BQDHWKLXFEN34OD7H7U6W4S4XO6A, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.975471667 (25h18m50.451410122s) [noise=true , address=45.91.116.5:4299 , piece_id=QU6ETEVLPAQY5YDF33435RD54MB6E27D2KWEMDARPPU5PWQMLT7Q, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.975630882 (25h18m50.451569347s) [noise=true , address=77.90.31.14:4663 , piece_id=RF6KXZ6KEX7SCTZC3DGRBXYVYGEL7PUXVOCWZTMKGSDJEL5UYFZA, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.975859649 (25h18m50.451798114s) [noise=true , address=104.36.82.4:4801 , piece_id=ORFMROERL57YQNCEIXTRWICOPNRTHVVZEOTCRTQI23VU6ECXH2RQ, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.976123605 (25h18m50.45206207s) [noise=true , address=104.36.82.7:4910 , piece_id=7YGVSFGWNC63F7MC6VXVCYXQDIGYNHSHMCVCVVCTJ2DFTYE2KWVA, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.976382981 (25h18m50.452321446s) [noise=true , address=45.91.116.14:4190 , piece_id=J26BTFI5U7AXWC4X2M2F2JHXN3QQD2C32QCU3S34HEJTOUNCF4UQ, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.976675641 (25h18m50.452614096s) [noise=true , address=45.91.116.6:4974 , piece_id=MIZHS2QYMIATZAD3H262GWSNXXAMP7V42VQEGU7SV7VJPPM7WQBQ, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.976878219 (25h18m50.452816684s) [noise=true , address=77.90.31.15:4699 , piece_id=5FQD72IB47PHRGSTYHFCDM53IK2KSP4VD322TAZA72YD636QVSLQ, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.977069153 (25h18m50.453007618s) [noise=true , address=104.36.82.4:4802 , piece_id=N7JMKKJFLXVINPDIOHEEBF3IY3MQ5JCYNFXQNI5CUGN7MXT5PALQ, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.977279286 (25h18m50.453217751s) [noise=true , address=77.90.31.10:4517 , piece_id=PBWUXJFF2KSYOIMVXKO2W3VREW3M6SIP6TBXZGGEKQTKBLAHVXNA, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.978133701 (25h18m50.454072176s) [noise=true , address=77.90.31.12:4590 , piece_id=V5GNOM2XCXMFOGC2LFVMO5BGACSW37BOEOBKHPQLY4TKSOCXYMVA, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.978578932 (25h18m50.454517397s) [noise=true , address=77.90.31.21:4878 , piece_id=35WEE5PRGXY4D3ZT5IFEXSS4UVAACRLEVTEXVIQNLBD3ZILOHPMA, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:37.978877754 (25h18m50.454816219s) [noise=true , address=77.90.31.19:4841 , piece_id=3KNKJM323L4KDLP35NKIS7HZGGU4YIOIHV54WMPOQY5IREVLLW6A, seg_pos={0 0}, upload=24231]: Uploading piece...
Jul 29 18:04:40.031482569 (25h18m52.507421034s) [upload=24231]: Next returned nil segment
Jul 29 18:04:40.031497318 (25h18m52.507435783s) [upload=24231]: Waiting for error group managing segments...
Jul 29 18:04:40.366807625 (25h18m52.84274609s) [noise=true , address=104.36.82.4:4802 , piece_id=N7JMKKJFLXVINPDIOHEEBF3IY3MQ5JCYNFXQNI5CUGN7MXT5PALQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.394982342 (25h18m52.870920827s) [noise=true , address=77.90.31.14:4663 , piece_id=RF6KXZ6KEX7SCTZC3DGRBXYVYGEL7PUXVOCWZTMKGSDJEL5UYFZA, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.418234261 (25h18m52.894172726s) [noise=true , address=45.91.116.10:4083 , piece_id=NO77ZLIGYMBJOW4XS2EW2VVWAJJBCZQV7JT66QGNPF4PVUMPAJ7A, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.495949506 (25h18m52.971887971s) [noise=true , address=104.36.82.4:4801 , piece_id=ORFMROERL57YQNCEIXTRWICOPNRTHVVZEOTCRTQI23VU6ECXH2RQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.502738256 (25h18m52.978676721s) [noise=true , address=77.90.31.21:4878 , piece_id=35WEE5PRGXY4D3ZT5IFEXSS4UVAACRLEVTEXVIQNLBD3ZILOHPMA, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.516995765 (25h18m52.99293423s) [noise=true , address=45.91.116.5:4299 , piece_id=QU6ETEVLPAQY5YDF33435RD54MB6E27D2KWEMDARPPU5PWQMLT7Q, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.518122691 (25h18m52.994061156s) [noise=true , address=45.91.116.6:4974 , piece_id=MIZHS2QYMIATZAD3H262GWSNXXAMP7V42VQEGU7SV7VJPPM7WQBQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.544864777 (25h18m53.020803242s) [noise=true , address=77.90.31.15:4699 , piece_id=5FQD72IB47PHRGSTYHFCDM53IK2KSP4VD322TAZA72YD636QVSLQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.578072334 (25h18m53.054010799s) [noise=true , address=77.90.31.19:4841 , piece_id=3KNKJM323L4KDLP35NKIS7HZGGU4YIOIHV54WMPOQY5IREVLLW6A, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.608525722 (25h18m53.084464187s) [noise=true , address=45.91.116.14:4190 , piece_id=J26BTFI5U7AXWC4X2M2F2JHXN3QQD2C32QCU3S34HEJTOUNCF4UQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.647568904 (25h18m53.123507369s) [noise=true , address=104.36.82.16:4295 , piece_id=CO4OE575M6EQXZP2KBPFLP57SQJMKGWKVT6E2HSS4GVACW75BECA, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:40.884037216 (25h18m53.359975681s) [noise=true , address=104.36.82.7:4910 , piece_id=7YGVSFGWNC63F7MC6VXVCYXQDIGYNHSHMCVCVVCTJ2DFTYE2KWVA, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:41.042983211 (25h18m53.518921676s) [noise=true , address=45.91.116.13:4154 , piece_id=YTYBO3FGNXYRCZLSNAKTUUT6BQDHWKLXFEN34OD7H7U6W4S4XO6A, seg_pos={0 0}, upload=24231]: Done uploading piece. err: <nil>
Jul 29 18:04:41.043003030 (25h18m53.518941495s) [seg_pos={0 0}, upload=24231]: Segment reached optimal threshold of 13 pieces.
Jul 29 18:04:41.043142627 (25h18m53.519081092s) [noise=true , address=77.90.31.10:4517 , piece_id=PBWUXJFF2KSYOIMVXKO2W3VREW3M6SIP6TBXZGGEKQTKBLAHVXNA, seg_pos={0 0}, upload=24231]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF
Jul 29 18:04:41.043333953 (25h18m53.519272418s) [noise=true , address=77.90.31.12:4590 , piece_id=V5GNOM2XCXMFOGC2LFVMO5BGACSW37BOEOBKHPQLY4TKSOCXYMVA, seg_pos={0 0}, upload=24231]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF
Jul 29 18:04:41.043381172 (25h18m53.519319627s) [noise=true , address=45.91.116.11:4118 , piece_id=GCJPBH7YYLU5V4KIMLMJ6GH67D5D43OIKRQX3YGMQWAKIAYHKNHQ, seg_pos={0 0}, upload=24231]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:56648->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:56648->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56648->45.91.116.11:4118: use of closed network connection
Jul 29 18:04:41.043389960 (25h18m53.519328425s) [seg_pos={0 0}, upload=24231]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:04:41.043545126 (25h18m53.519483591s) [upload=24231]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:04:41.043550778 (25h18m53.519489243s) [upload=24231]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:04:41.465788927 (25h18m53.941727392s) [upload=24231]: Done uploading segments...
Jul 29 18:04:41.465842159 (25h18m53.941780624s) [upload=24231]: Upload finished. err: <nil>
Duration: 30.002470283s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X...X.................................X.................] duration:
1.900530248s
FAIL (2025-07-29 18:04:56 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.900554635s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
8 out of 60 is failed (threshold=3) [.................................X..........X.XX......XX.XX.] duration:
23.650023131s
FAIL (2025-07-29 18:04:05 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 23.650041428s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.001669267s
FAIL (2025-07-29 18:04:02 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
d cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: read tcp 188.166.225.170:53002->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 29 18:02:52.110651937 (25h17m4.586590392s) [seg_pos={0 0}, upload=24201]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:02:52.110720287 (25h17m4.586658752s) [upload=24201]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:02:52.110726098 (25h17m4.586664563s) [upload=24201]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:02:52.412599811 (25h17m4.888538276s) [upload=24201]: Done uploading segments...
Jul 29 18:02:52.412616774 (25h17m4.888555239s) [upload=24201]: Upload finished. err: <nil>
Jul 29 18:03:32.675748529 (25h17m45.151686994s) [upload=24215]: Starting upload
Jul 29 18:03:32.675808925 (25h17m45.15174739s) [upload=24215]: Done starting upload
Jul 29 18:03:32.675922192 (25h17m45.151860657s) [upload=24215]: Uploading segments...
Jul 29 18:03:32.676075284 (25h17m45.152013749s) [upload=24215]: Got next segment. Inline: false
Jul 29 18:03:32.676139868 (25h17m45.152078333s) [upload=24215]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 18:03:32.676147022 (25h17m45.152085487s) [upload=24215]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 18:03:32.911580854 (25h17m45.387519309s) [seg_pos={0 0}, upload=24215]: Begin upload segment...
Jul 29 18:03:32.911632994 (25h17m45.387571449s) [seg_pos={0 0}, upload=24215]: Done begin upload segment.
Jul 29 18:03:32.911678530 (25h17m45.387616995s) [noise=true , address=45.91.116.8:4047 , piece_id=6EKF72ODMOHJH5GCRQR4Z74Z5N73ENMCGWL3Y4637HE5TD2YOTSA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.911719850 (25h17m45.387658315s) [noise=true , address=104.36.82.5:4839 , piece_id=LM3RSOA23L3RP2DPXXOR2WEURLS2MBHU6K5NKC4KMZDMDOYWIGUA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912052666 (25h17m45.387991132s) [noise=true , address=77.90.31.16:4733 , piece_id=EK3KUUONL5KH6WP6DRXS57QOEXX5CKOLI6CDHKIQHUKWRUE5B3HA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912269612 (25h17m45.388208077s) [noise=true , address=45.91.116.12:4226 , piece_id=Z2FF4UONSCW4KRW4TS267LID4DOO5COMUM6EZIBYRLHKEX7GKEAA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912411012 (25h17m45.388349477s) [noise=true , address=104.36.82.15:4259 , piece_id=TDDBQOLI2L6IFNE6H6FRHWFY65SJ4D747XBSBGBUF45XDL5ACADQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912581638 (25h17m45.388520103s) [noise=true , address=104.36.82.6:4873 , piece_id=FDUMN5KUBQGIUYOU6KFWEJ3IGMQKLDL4Y6HH25WU5OKFIWIFOTAQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912668094 (25h17m45.388606559s) [noise=true , address=45.91.116.14:4189 , piece_id=BHKGEYQZYXRJZOON2XDBZMFUQHHENYNZQMINUSLPBN2PP24WB2AQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912883105 (25h17m45.38882156s) [noise=true , address=104.36.82.10:4077 , piece_id=VTPEREFXUKP2TURLPENV75DDVDQVZT6JC4JVQD6AFBV7EXLK25VA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.912950495 (25h17m45.38888895s) [noise=true , address=77.90.31.10:4520 , piece_id=J2OEF4N3VAA4HV4ONFL5RXV2JCT2MPK3DZGLUYBWM4XQWFDCJ4AA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.913168100 (25h17m45.389106565s) [noise=true , address=104.36.82.4:4804 , piece_id=O5Q7Q2YLWFHVBJM74FQ2NRZ6OK5GJZJNSP7VOURUO76ZP2DH4ISA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.913208839 (25h17m45.389147304s) [noise=true , address=77.90.31.16:4735 , piece_id=YCCPGRCS6XONEW7SZZQTX4JVAD2U32XQ6K53O7FLMQOFZOXTMIBQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.913421706 (25h17m45.389360171s) [noise=true , address=45.91.116.6:4973 , piece_id=L6P2QZLZE5HFD7TEQDTV6VQ2Q3BOU2OEBP7FBATFOMEPOT7UMSVQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.913430613 (25h17m45.389369078s) [noise=true , address=104.36.82.6:4876 , piece_id=OWW2CNPWMGOBNWFVHRI5YMGQUAWOSERZU4HWSUFKRV4C3PUOZFUQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.913622530 (25h17m45.389560995s) [noise=true , address=77.90.31.11:4555 , piece_id=7C7QX7OVCES46SVZ3MOPYQ2JRPH4D4GRY3GIZH3NTMRM6AB57APA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.913993910 (25h17m45.389932365s) [noise=true , address=45.91.116.10:4082 , piece_id=QE7YSXR4WW2YCEYBMA7AEIUS6I6IS3ZSWZUT6UAK3DUS3XEOPUBA, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:32.914240622 (25h17m45.390179087s) [noise=true , address=104.36.82.7:4911 , piece_id=EEFCB7Y2R2FXRRKVTXEUKSGPD7GBXUE6FSP4R6T27YFBQLW7ALTQ, seg_pos={0 0}, upload=24215]: Uploading piece...
Jul 29 18:03:34.852899372 (25h17m47.328837847s) [upload=24215]: Next returned nil segment
Jul 29 18:03:34.852917637 (25h17m47.328856112s) [upload=24215]: Waiting for error group managing segments...
Jul 29 18:03:35.192373367 (25h17m47.668311822s) [noise=true , address=45.91.116.14:4189 , piece_id=BHKGEYQZYXRJZOON2XDBZMFUQHHENYNZQMINUSLPBN2PP24WB2AQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.201038616 (25h17m47.676977081s) [noise=true , address=45.91.116.10:4082 , piece_id=QE7YSXR4WW2YCEYBMA7AEIUS6I6IS3ZSWZUT6UAK3DUS3XEOPUBA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.257516757 (25h17m47.733455222s) [noise=true , address=77.90.31.11:4555 , piece_id=7C7QX7OVCES46SVZ3MOPYQ2JRPH4D4GRY3GIZH3NTMRM6AB57APA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.269204950 (25h17m47.745143405s) [noise=true , address=45.91.116.8:4047 , piece_id=6EKF72ODMOHJH5GCRQR4Z74Z5N73ENMCGWL3Y4637HE5TD2YOTSA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.299493613 (25h17m47.775432078s) [noise=true , address=77.90.31.16:4733 , piece_id=EK3KUUONL5KH6WP6DRXS57QOEXX5CKOLI6CDHKIQHUKWRUE5B3HA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.307942268 (25h17m47.783880733s) [noise=true , address=45.91.116.6:4973 , piece_id=L6P2QZLZE5HFD7TEQDTV6VQ2Q3BOU2OEBP7FBATFOMEPOT7UMSVQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.338429179 (25h17m47.814367644s) [noise=true , address=77.90.31.16:4735 , piece_id=YCCPGRCS6XONEW7SZZQTX4JVAD2U32XQ6K53O7FLMQOFZOXTMIBQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.338635444 (25h17m47.814573939s) [noise=true , address=104.36.82.7:4911 , piece_id=EEFCB7Y2R2FXRRKVTXEUKSGPD7GBXUE6FSP4R6T27YFBQLW7ALTQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.347814566 (25h17m47.823753031s) [noise=true , address=45.91.116.12:4226 , piece_id=Z2FF4UONSCW4KRW4TS267LID4DOO5COMUM6EZIBYRLHKEX7GKEAA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.388578842 (25h17m47.864517317s) [noise=true , address=104.36.82.6:4873 , piece_id=FDUMN5KUBQGIUYOU6KFWEJ3IGMQKLDL4Y6HH25WU5OKFIWIFOTAQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.698489473 (25h17m48.174427938s) [noise=true , address=104.36.82.5:4839 , piece_id=LM3RSOA23L3RP2DPXXOR2WEURLS2MBHU6K5NKC4KMZDMDOYWIGUA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:35.770808944 (25h17m48.246747409s) [noise=true , address=77.90.31.10:4520 , piece_id=J2OEF4N3VAA4HV4ONFL5RXV2JCT2MPK3DZGLUYBWM4XQWFDCJ4AA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:36.280597987 (25h17m48.756536452s) [noise=true , address=104.36.82.4:4804 , piece_id=O5Q7Q2YLWFHVBJM74FQ2NRZ6OK5GJZJNSP7VOURUO76ZP2DH4ISA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: <nil>
Jul 29 18:03:36.280624437 (25h17m48.756562902s) [seg_pos={0 0}, upload=24215]: Segment reached optimal threshold of 13 pieces.
Jul 29 18:03:36.280732003 (25h17m48.756670468s) [noise=true , address=104.36.82.10:4077 , piece_id=VTPEREFXUKP2TURLPENV75DDVDQVZT6JC4JVQD6AFBV7EXLK25VA, seg_pos={0 0}, upload=24215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF
Jul 29 18:03:36.280824951 (25h17m48.756763406s) [noise=true , address=104.36.82.15:4259 , piece_id=TDDBQOLI2L6IFNE6H6FRHWFY65SJ4D747XBSBGBUF45XDL5ACADQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF
Jul 29 18:03:36.280883753 (25h17m48.756822218s) [noise=true , address=104.36.82.6:4876 , piece_id=OWW2CNPWMGOBNWFVHRI5YMGQUAWOSERZU4HWSUFKRV4C3PUOZFUQ, seg_pos={0 0}, upload=24215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: write tcp 188.166.225.170:50358->104.36.82.6:4876: use of closed network connection; write tcp 188.166.225.170:50358->104.36.82.6:4876: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50358->104.36.82.6:4876: use of closed network connection
Jul 29 18:03:36.280903070 (25h17m48.756841535s) [seg_pos={0 0}, upload=24215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:03:36.280964928 (25h17m48.756903393s) [upload=24215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:03:36.280968505 (25h17m48.75690697s) [upload=24215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:03:36.580401990 (25h17m49.056340455s) [upload=24215]: Done uploading segments...
Jul 29 18:03:36.580420275 (25h17m49.05635874s) [upload=24215]: Upload finished. err: <nil>
Duration: 30.00168116s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....X...X.................................X................] duration:
1.828386773s
FAIL (2025-07-29 18:03:50 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.828410648s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
8 out of 60 is failed (threshold=3) [..................................X..........X.XX......XX.XX] duration:
30.002288634s
FAIL (2025-07-29 18:03:22 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 30.002313432s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
30.003336071s
FAIL (2025-07-29 18:03:16 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
1755 (25h16m19.79653023s) [noise=true , address=77.90.31.18:4805 , piece_id=BX63WCO4NDP6PDYBELQBQE2DMUIC7Z7OX6OLYLHRWD732RPXK4ZQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: read tcp 188.166.225.170:38132->77.90.31.18:4805: use of closed network connection; context canceled
Jul 29 18:02:07.320611934 (25h16m19.796550409s) [seg_pos={0 0}, upload=24193]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:02:07.320671067 (25h16m19.796609532s) [upload=24193]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:02:07.320675585 (25h16m19.79661405s) [upload=24193]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:02:07.623972307 (25h16m20.099910782s) [upload=24193]: Done uploading segments...
Jul 29 18:02:07.623994831 (25h16m20.099933296s) [upload=24193]: Upload finished. err: <nil>
Jul 29 18:02:47.581456532 (25h17m0.057394997s) [upload=24201]: Starting upload
Jul 29 18:02:47.581514473 (25h17m0.057452938s) [upload=24201]: Done starting upload
Jul 29 18:02:47.581588575 (25h17m0.05752703s) [upload=24201]: Uploading segments...
Jul 29 18:02:47.581700659 (25h17m0.057639124s) [upload=24201]: Got next segment. Inline: false
Jul 29 18:02:47.581734173 (25h17m0.057672638s) [upload=24201]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 18:02:47.581738000 (25h17m0.057676456s) [upload=24201]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 18:02:47.813336253 (25h17m0.289274719s) [seg_pos={0 0}, upload=24201]: Begin upload segment...
Jul 29 18:02:47.813402290 (25h17m0.289340755s) [seg_pos={0 0}, upload=24201]: Done begin upload segment.
Jul 29 18:02:47.813434903 (25h17m0.289373358s) [noise=true , address=45.91.116.9:4261 , piece_id=7MQIMB53S2T3Z62T5COC7QVR2FDUGEQLHOXP3YAB4KM77DIZ5NRA, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.813898069 (25h17m0.289836524s) [noise=true , address=77.90.31.14:4662 , piece_id=E77QF74YCX7ZLLIRBQMRWYT6QC5K3IZE75DF45WFJ2K7SAKHBPAQ, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.813905222 (25h17m0.289843687s) [noise=true , address=77.90.31.10:4517 , piece_id=IUTBPLYGZLCVOEPEOUL65GBW7NM6CX6ZKAB4DVTZRI7X56UOVL7Q, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814197452 (25h17m0.290135917s) [noise=true , address=77.90.31.16:4734 , piece_id=JNKGNHZMLMGZBQDXKAUT2XM2ZIWE6ELGFY7ZJOD4SWKRHRYIOTZA, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814264850 (25h17m0.290203315s) [noise=true , address=45.91.116.10:4081 , piece_id=C2JWQABLH5CLALYONYUBG2J7GI7C24TLXH6IH3WDR4LX2TJU2J6Q, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814459433 (25h17m0.290397898s) [noise=true , address=77.90.31.11:4553 , piece_id=PH5FWH6URCFKHNZLUDWJ72VIANJ4JOZSCTAXRCVEUKLP5BZSPDLQ, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814515410 (25h17m0.290453875s) [noise=true , address=45.91.116.8:4045 , piece_id=YQLDNFYF47BX4XNWKSLMO5JKB23JXKQ7C5NJV7ZC6G3EH6225O7A, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814673703 (25h17m0.290612168s) [noise=true , address=45.91.116.13:4155 , piece_id=37MQITPII4UZJFR4VVK4FKSFKHXU3DR6FQOPDGXW3YKBWBQITXVQ, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814739839 (25h17m0.290678304s) [noise=true , address=45.91.116.12:4225 , piece_id=YBLAUOPJTMVGJX5CLK5EVK3SQT7MVBNQWJPLXOB34FSPWFSW5ECQ, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814891449 (25h17m0.290829915s) [noise=true , address=104.36.82.4:4801 , piece_id=VEFUN34TXVTD3KUUVRUCER2ABYQAWF5XX4DOE6KKMLYOX7YKOTIQ, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.814962956 (25h17m0.290901421s) [noise=true , address=104.36.82.7:4909 , piece_id=QUOBHEOANNYSQ5M7SUVZACCVERWURH3QBPRX7BYDF5DKXJNHNQYA, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.815174841 (25h17m0.291113296s) [noise=true , address=77.90.31.17:4770 , piece_id=5RCMOQ626ZDHESS5JWPD574VH2HGIKB2B2T3DG5Q465RHOEWEU4Q, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.815401385 (25h17m0.29133985s) [noise=true , address=45.91.116.6:4975 , piece_id=IPREW3SYDRNM2E3DK3XKHEXYCXPHB22SK35JTJR2V4GKKVNK7APA, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.815616927 (25h17m0.291555392s) [noise=true , address=45.91.116.4:4939 , piece_id=47D4H5AYIY6VRI4DMCOJUXHUBDQ5DLEYPN7OICE4HBG5ZBSQNY5Q, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.815860073 (25h17m0.291798538s) [noise=true , address=77.90.31.19:4843 , piece_id=C7MFEBKFDP5ZUDPKXLL7G2ZYC7Q7M4TFHQHGD7YHBYBGFHAJ6YRQ, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:47.816018005 (25h17m0.29195646s) [noise=true , address=77.90.31.16:4735 , piece_id=NMNGSAFK5OQYDGSZJX6A5UFLMXXKWTCZW3MHOF6OO2QQIQAJJP6A, seg_pos={0 0}, upload=24201]: Uploading piece...
Jul 29 18:02:49.878735048 (25h17m2.354673513s) [upload=24201]: Next returned nil segment
Jul 29 18:02:49.878744987 (25h17m2.354683452s) [upload=24201]: Waiting for error group managing segments...
Jul 29 18:02:50.205526897 (25h17m2.681465362s) [noise=true , address=77.90.31.16:4735 , piece_id=NMNGSAFK5OQYDGSZJX6A5UFLMXXKWTCZW3MHOF6OO2QQIQAJJP6A, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.281465633 (25h17m2.757404098s) [noise=true , address=104.36.82.7:4909 , piece_id=QUOBHEOANNYSQ5M7SUVZACCVERWURH3QBPRX7BYDF5DKXJNHNQYA, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.306254934 (25h17m2.782193399s) [noise=true , address=77.90.31.16:4734 , piece_id=JNKGNHZMLMGZBQDXKAUT2XM2ZIWE6ELGFY7ZJOD4SWKRHRYIOTZA, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.472333326 (25h17m2.948271791s) [noise=true , address=77.90.31.10:4517 , piece_id=IUTBPLYGZLCVOEPEOUL65GBW7NM6CX6ZKAB4DVTZRI7X56UOVL7Q, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.524208110 (25h17m3.000146575s) [noise=true , address=45.91.116.8:4045 , piece_id=YQLDNFYF47BX4XNWKSLMO5JKB23JXKQ7C5NJV7ZC6G3EH6225O7A, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.575210433 (25h17m3.051148899s) [noise=true , address=45.91.116.9:4261 , piece_id=7MQIMB53S2T3Z62T5COC7QVR2FDUGEQLHOXP3YAB4KM77DIZ5NRA, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.700193467 (25h17m3.176131932s) [noise=true , address=45.91.116.12:4225 , piece_id=YBLAUOPJTMVGJX5CLK5EVK3SQT7MVBNQWJPLXOB34FSPWFSW5ECQ, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:50.759045240 (25h17m3.234983705s) [noise=true , address=77.90.31.19:4843 , piece_id=C7MFEBKFDP5ZUDPKXLL7G2ZYC7Q7M4TFHQHGD7YHBYBGFHAJ6YRQ, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:51.040974142 (25h17m3.516912607s) [noise=true , address=77.90.31.11:4553 , piece_id=PH5FWH6URCFKHNZLUDWJ72VIANJ4JOZSCTAXRCVEUKLP5BZSPDLQ, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:51.120250393 (25h17m3.596188858s) [noise=true , address=45.91.116.10:4081 , piece_id=C2JWQABLH5CLALYONYUBG2J7GI7C24TLXH6IH3WDR4LX2TJU2J6Q, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:51.181468704 (25h17m3.657407169s) [noise=true , address=77.90.31.14:4662 , piece_id=E77QF74YCX7ZLLIRBQMRWYT6QC5K3IZE75DF45WFJ2K7SAKHBPAQ, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:51.390231746 (25h17m3.866170212s) [noise=true , address=45.91.116.4:4939 , piece_id=47D4H5AYIY6VRI4DMCOJUXHUBDQ5DLEYPN7OICE4HBG5ZBSQNY5Q, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:52.110173581 (25h17m4.586112046s) [noise=true , address=77.90.31.17:4770 , piece_id=5RCMOQ626ZDHESS5JWPD574VH2HGIKB2B2T3DG5Q465RHOEWEU4Q, seg_pos={0 0}, upload=24201]: Done uploading piece. err: <nil>
Jul 29 18:02:52.110201895 (25h17m4.58614036s) [seg_pos={0 0}, upload=24201]: Segment reached optimal threshold of 13 pieces.
Jul 29 18:02:52.110304832 (25h17m4.586243297s) [noise=true , address=45.91.116.6:4975 , piece_id=IPREW3SYDRNM2E3DK3XKHEXYCXPHB22SK35JTJR2V4GKKVNK7APA, seg_pos={0 0}, upload=24201]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF
Jul 29 18:02:52.110400436 (25h17m4.586338901s) [noise=true , address=104.36.82.4:4801 , piece_id=VEFUN34TXVTD3KUUVRUCER2ABYQAWF5XX4DOE6KKMLYOX7YKOTIQ, seg_pos={0 0}, upload=24201]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 29 18:02:52.110639983 (25h17m4.586578448s) [noise=true , address=45.91.116.13:4155 , piece_id=37MQITPII4UZJFR4VVK4FKSFKHXU3DR6FQOPDGXW3YKBWBQITXVQ, seg_pos={0 0}, upload=24201]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: read tcp 188.166.225.170:53002->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 29 18:02:52.110651937 (25h17m4.586590392s) [seg_pos={0 0}, upload=24201]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:02:52.110720287 (25h17m4.586658752s) [upload=24201]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:02:52.110726098 (25h17m4.586664563s) [upload=24201]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:02:52.412599811 (25h17m4.888538276s) [upload=24201]: Done uploading segments...
Jul 29 18:02:52.412616774 (25h17m4.888555239s) [upload=24201]: Upload finished. err: <nil>
Duration: 30.003370097s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X...X.................................X...............] duration:
1.831791038s
FAIL (2025-07-29 18:03:01 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.831821276s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
30.002185094s
FAIL (2025-07-29 18:02:32 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
ed piece hash; context canceled; EOF
Jul 29 18:00:59.304575292 (25h15m11.780513757s) [noise=true , address=104.36.82.4:4804 , piece_id=TGOL5VOG4KVW7GRA5JDT7R54SAJBBQTYKP6UVGUHWK4NS4YYF2UQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 29 18:00:59.304606281 (25h15m11.780544746s) [seg_pos={0 0}, upload=24174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:00:59.304686615 (25h15m11.78062508s) [upload=24174]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:00:59.304698428 (25h15m11.780636893s) [upload=24174]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:00:59.601530762 (25h15m12.077469217s) [upload=24174]: Done uploading segments...
Jul 29 18:00:59.601549638 (25h15m12.077488103s) [upload=24174]: Upload finished. err: <nil>
Jul 29 18:02:03.277790568 (25h16m15.753729063s) [upload=24193]: Starting upload
Jul 29 18:02:03.277946497 (25h16m15.753884962s) [upload=24193]: Done starting upload
Jul 29 18:02:03.278038041 (25h16m15.753976557s) [upload=24193]: Uploading segments...
Jul 29 18:02:03.278358214 (25h16m15.754296699s) [upload=24193]: Got next segment. Inline: false
Jul 29 18:02:03.278407838 (25h16m15.754346354s) [upload=24193]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 18:02:03.278415744 (25h16m15.754354249s) [upload=24193]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 18:02:03.512194516 (25h16m15.988132981s) [seg_pos={0 0}, upload=24193]: Begin upload segment...
Jul 29 18:02:03.512228681 (25h16m15.988167136s) [seg_pos={0 0}, upload=24193]: Done begin upload segment.
Jul 29 18:02:03.512477597 (25h16m15.988416062s) [noise=true , address=45.91.116.14:4189 , piece_id=Y2MRUPITO5MUKD4Z5IYQVNZ6AK2OKCHPPRZJEZGUY32UQKNGXU7A, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.512573049 (25h16m15.988511515s) [noise=true , address=104.36.82.5:4838 , piece_id=QIU4I54LHUYCMSF7ZD4JH7XE53C543YOC4VADYKRCLNLZCRSOFZQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.512947877 (25h16m15.988886342s) [noise=true , address=45.91.116.8:4047 , piece_id=DVO6DKVW7KIX5XH35D5EVRAK327VEW4JLBF5XUSTG4PN2IXMA7YQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.513505975 (25h16m15.98944444s) [noise=true , address=45.91.116.6:4975 , piece_id=4MP23QE25ENZXU6PCWLYCVKJAZVVMAASYVRPK3YHPPYW2DE7ZM4Q, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.513968319 (25h16m15.989906784s) [noise=true , address=45.91.116.6:4974 , piece_id=HDNRT5PZ77JUZPO652VQIM2PGIVAMT7JGKFJIU4UFOPRSEZTFUFA, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.514545323 (25h16m15.990483788s) [noise=true , address=77.90.31.10:4519 , piece_id=R2RULXDUZNNLS3XTWSKWITD2PCT3AFKAMUP4XQGDCE2TLOUE4AUQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.514760245 (25h16m15.99069871s) [noise=true , address=104.36.82.16:4295 , piece_id=V6SO7HNFSULY37NAIITSEOBZKJUP3LUD3RNYS4PXOFDNTKY6HMVQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.514972711 (25h16m15.990911177s) [noise=true , address=45.91.116.5:4298 , piece_id=PUTCELC4VB2GVEDXL2UY7R6ZWKYDRQIJIL7YNDAZRIAKA3N4BDHA, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.515518676 (25h16m15.991457141s) [noise=true , address=45.91.116.4:4937 , piece_id=6XLNZXPGVSIRZO7TTFGBECM6CJAWXWCOABXWY7IAMVQ6WKMTRBCQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.515725361 (25h16m15.991663826s) [noise=true , address=77.90.31.18:4805 , piece_id=BX63WCO4NDP6PDYBELQBQE2DMUIC7Z7OX6OLYLHRWD732RPXK4ZQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.515932468 (25h16m15.991870933s) [noise=true , address=77.90.31.13:4627 , piece_id=AQVNKBM5RQVITLOEWQ4T24XEYNQ3Y3MXN5WAWNHJD23ICPECGXOA, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.516131559 (25h16m15.992070024s) [noise=true , address=45.91.116.11:4119 , piece_id=SBDML6ZYPZQCSY755PPNA4OX3QILCLEF5RF6E4T45YLM42V2Z52A, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.516370686 (25h16m15.992309141s) [noise=true , address=104.36.82.15:4257 , piece_id=VM574RRRCWAGCAGL24GVYB6T2CATIE4RJSQCZ5MFHQMDTUF4OWYA, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.516989790 (25h16m15.992928255s) [noise=true , address=104.36.82.6:4874 , piece_id=B72722YLH5MU44DCA4QBK2RGV45LLZSLNLT322NDZQ7HJKVNINNQ, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.517221454 (25h16m15.993159919s) [noise=true , address=45.91.116.12:4227 , piece_id=MHXO7YDVOP2I4OBLYVCEKDJR47EBCEAFDODPKLHBDOGFBSGNT2ZA, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:03.517474288 (25h16m15.993412753s) [noise=true , address=77.90.31.16:4734 , piece_id=XDRUGT5OUDXNXQKFE7RX4TLVH7VSWM6544MLPXF2BZWOH4GK4XZA, seg_pos={0 0}, upload=24193]: Uploading piece...
Jul 29 18:02:05.453906902 (25h16m17.929845367s) [upload=24193]: Next returned nil segment
Jul 29 18:02:05.453922681 (25h16m17.929861146s) [upload=24193]: Waiting for error group managing segments...
Jul 29 18:02:06.050848167 (25h16m18.526786633s) [noise=true , address=45.91.116.11:4119 , piece_id=SBDML6ZYPZQCSY755PPNA4OX3QILCLEF5RF6E4T45YLM42V2Z52A, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.062863767 (25h16m18.538802252s) [noise=true , address=77.90.31.10:4519 , piece_id=R2RULXDUZNNLS3XTWSKWITD2PCT3AFKAMUP4XQGDCE2TLOUE4AUQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.074323032 (25h16m18.550261497s) [noise=true , address=45.91.116.8:4047 , piece_id=DVO6DKVW7KIX5XH35D5EVRAK327VEW4JLBF5XUSTG4PN2IXMA7YQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.076303010 (25h16m18.552241476s) [noise=true , address=77.90.31.16:4734 , piece_id=XDRUGT5OUDXNXQKFE7RX4TLVH7VSWM6544MLPXF2BZWOH4GK4XZA, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.077899285 (25h16m18.55383775s) [noise=true , address=45.91.116.6:4975 , piece_id=4MP23QE25ENZXU6PCWLYCVKJAZVVMAASYVRPK3YHPPYW2DE7ZM4Q, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.080375061 (25h16m18.556313516s) [noise=true , address=104.36.82.5:4838 , piece_id=QIU4I54LHUYCMSF7ZD4JH7XE53C543YOC4VADYKRCLNLZCRSOFZQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.080968207 (25h16m18.556906672s) [noise=true , address=104.36.82.6:4874 , piece_id=B72722YLH5MU44DCA4QBK2RGV45LLZSLNLT322NDZQ7HJKVNINNQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.112120711 (25h16m18.588059176s) [noise=true , address=104.36.82.15:4257 , piece_id=VM574RRRCWAGCAGL24GVYB6T2CATIE4RJSQCZ5MFHQMDTUF4OWYA, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.123284991 (25h16m18.599223456s) [noise=true , address=45.91.116.4:4937 , piece_id=6XLNZXPGVSIRZO7TTFGBECM6CJAWXWCOABXWY7IAMVQ6WKMTRBCQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.862022940 (25h16m19.337961405s) [noise=true , address=45.91.116.12:4227 , piece_id=MHXO7YDVOP2I4OBLYVCEKDJR47EBCEAFDODPKLHBDOGFBSGNT2ZA, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:06.991312821 (25h16m19.467251286s) [noise=true , address=77.90.31.13:4627 , piece_id=AQVNKBM5RQVITLOEWQ4T24XEYNQ3Y3MXN5WAWNHJD23ICPECGXOA, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:07.174844294 (25h16m19.650782769s) [noise=true , address=45.91.116.5:4298 , piece_id=PUTCELC4VB2GVEDXL2UY7R6ZWKYDRQIJIL7YNDAZRIAKA3N4BDHA, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:07.320280981 (25h16m19.796219446s) [noise=true , address=104.36.82.16:4295 , piece_id=V6SO7HNFSULY37NAIITSEOBZKJUP3LUD3RNYS4PXOFDNTKY6HMVQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: <nil>
Jul 29 18:02:07.320308944 (25h16m19.796247409s) [seg_pos={0 0}, upload=24193]: Segment reached optimal threshold of 13 pieces.
Jul 29 18:02:07.320425747 (25h16m19.796364222s) [noise=true , address=45.91.116.6:4974 , piece_id=HDNRT5PZ77JUZPO652VQIM2PGIVAMT7JGKFJIU4UFOPRSEZTFUFA, seg_pos={0 0}, upload=24193]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF
Jul 29 18:02:07.320453911 (25h16m19.796392446s) [noise=true , address=45.91.116.14:4189 , piece_id=Y2MRUPITO5MUKD4Z5IYQVNZ6AK2OKCHPPRZJEZGUY32UQKNGXU7A, seg_pos={0 0}, upload=24193]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF
Jul 29 18:02:07.320591755 (25h16m19.79653023s) [noise=true , address=77.90.31.18:4805 , piece_id=BX63WCO4NDP6PDYBELQBQE2DMUIC7Z7OX6OLYLHRWD732RPXK4ZQ, seg_pos={0 0}, upload=24193]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: read tcp 188.166.225.170:38132->77.90.31.18:4805: use of closed network connection; context canceled
Jul 29 18:02:07.320611934 (25h16m19.796550409s) [seg_pos={0 0}, upload=24193]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:02:07.320671067 (25h16m19.796609532s) [upload=24193]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:02:07.320675585 (25h16m19.79661405s) [upload=24193]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:02:07.623972307 (25h16m20.099910782s) [upload=24193]: Done uploading segments...
Jul 29 18:02:07.623994831 (25h16m20.099933296s) [upload=24193]: Upload finished. err: <nil>
Duration: 30.00220961s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [...................................X..........X.XX......XX.X] duration:
30.002189338s
FAIL (2025-07-29 18:02:17 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 30.002212722s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X...X.................................X..............] duration:
1.928878089s
FAIL (2025-07-29 18:02:05 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.928900111s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.002135371s
FAIL (2025-07-29 18:01:24 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
6.225.170:50486->104.36.82.7:4909: use of closed network connection
Jul 29 17:59:53.738754416 (25h14m6.214692881s) [noise=true , address=104.36.82.2:4732 , piece_id=WI2PQ2BNUURPDCCF6EXOYUEGGLZCJZE7UZQLHOUXQCSYXVKKOMGA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: read tcp 188.166.225.170:36902->104.36.82.2:4732: use of closed network connection; context canceled
Jul 29 17:59:53.738814050 (25h14m6.214752535s) [seg_pos={0 0}, upload=24155]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 17:59:53.738917608 (25h14m6.214856074s) [upload=24155]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 17:59:53.738925573 (25h14m6.214864038s) [upload=24155]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 17:59:54.021266822 (25h14m6.497205287s) [upload=24155]: Done uploading segments...
Jul 29 17:59:54.021286999 (25h14m6.497225464s) [upload=24155]: Upload finished. err: <nil>
Jul 29 18:00:55.535052666 (25h15m8.010991131s) [upload=24174]: Starting upload
Jul 29 18:00:55.535119995 (25h15m8.01105846s) [upload=24174]: Done starting upload
Jul 29 18:00:55.535216520 (25h15m8.011154985s) [upload=24174]: Uploading segments...
Jul 29 18:00:55.535314717 (25h15m8.011253172s) [upload=24174]: Got next segment. Inline: false
Jul 29 18:00:55.535366356 (25h15m8.011304821s) [upload=24174]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 18:00:55.535371376 (25h15m8.011309841s) [upload=24174]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 18:00:55.766594690 (25h15m8.242533195s) [seg_pos={0 0}, upload=24174]: Begin upload segment...
Jul 29 18:00:55.766665586 (25h15m8.242604091s) [seg_pos={0 0}, upload=24174]: Done begin upload segment.
Jul 29 18:00:55.766689051 (25h15m8.242627516s) [noise=true , address=45.91.116.7:4011 , piece_id=EEHSI72I5G4ICXNQBA5USWETF3IWHXKUMWHJPWT757NHHOLH542Q, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.766692988 (25h15m8.242631463s) [noise=true , address=104.36.82.14:4221 , piece_id=DXWRB2WXQ6YFQH4BORZWJJ2QCRM6X5KHAZPZWSUBXJQ2GWXFEOPA, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.767046815 (25h15m8.24298528s) [noise=true , address=45.91.116.11:4119 , piece_id=6JGY6CEHPGQILT774XMHHOWSAFGTDRBPYWF5ATRCYBZ4O6KH5VGQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.767346257 (25h15m8.243284722s) [noise=true , address=104.36.82.4:4804 , piece_id=TGOL5VOG4KVW7GRA5JDT7R54SAJBBQTYKP6UVGUHWK4NS4YYF2UQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.767600975 (25h15m8.24353943s) [noise=true , address=77.90.31.10:4517 , piece_id=KSERBGMKK2LYDTGS7LAPCGYTBLBTIR26AG5KKJP4JTVHFNDMTNRQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.768031068 (25h15m8.243969533s) [noise=true , address=45.91.116.14:4189 , piece_id=RUGRHEVJUBNXHEAJF3QCHX5J7VXCP6HB3YFMAAZ6MCYSUN37Z2SQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.768388973 (25h15m8.244327438s) [noise=true , address=45.91.116.6:4974 , piece_id=G53ZF3N5QARPXZBMFYBEGN3BIWIS72C5TTFNPGC4ROYSI2O72E4Q, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.768751307 (25h15m8.244689772s) [noise=true , address=45.91.116.7:4009 , piece_id=WR7GIDXACT2DDBO3RAZY2F6EEW53ACA4ZREPP2MKZMME6L2K7IRA, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.769049728 (25h15m8.244988193s) [noise=true , address=45.91.116.10:4083 , piece_id=LT7Y4QUQBSGYGQB5G5BRU55MWEQRB5KCPYHSUXQCRSHOYJIUKRRA, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.769247225 (25h15m8.24518569s) [noise=true , address=104.36.82.7:4911 , piece_id=SU7LULMMHKTQEDXW7JOFYNHZLZCRWAWM63WBNNZ5QKQI4JXEXEYQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.769461485 (25h15m8.245399951s) [noise=true , address=77.90.31.13:4626 , piece_id=3IJ3VRZXETGDTMEJD2GP62ZOSRB52CKU3LTIJVJT5RY4TUL4CKRQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.769652621 (25h15m8.245591086s) [noise=true , address=104.36.82.16:4294 , piece_id=ZLCMQWXUUCTBPOJGY6IJAP3KXG4AQMYIVFGPG36MSSOE2CQE3GGQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.769876630 (25h15m8.245815095s) [noise=true , address=45.91.116.4:4940 , piece_id=JLNQLKSX4QAAHFFE7Z6WJGHJRQ43OBCNQWVHKUSFQ2LBP6NGI2XQ, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.770081712 (25h15m8.246020177s) [noise=true , address=77.90.31.17:4772 , piece_id=MSRKPWDE4SRVTMMJYJPY5SIABTXWU3GCF6DUDRAKCEMCC4P64HKA, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.770302103 (25h15m8.246240578s) [noise=true , address=45.91.116.8:4045 , piece_id=H3PSQ2E33P5HA5SALHCHHXL3OGXMTINHC4ABU32IDZGZEMLOW34Q, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:55.770435579 (25h15m8.246374044s) [noise=true , address=104.36.82.2:4729 , piece_id=7S2VDHNG26CGKFVARA5MGE33D5BDA5PUQ4PBQQV4RIMMVNLUXSGA, seg_pos={0 0}, upload=24174]: Uploading piece...
Jul 29 18:00:57.568361920 (25h15m10.044300395s) [upload=24174]: Next returned nil segment
Jul 29 18:00:57.568382188 (25h15m10.044320663s) [upload=24174]: Waiting for error group managing segments...
Jul 29 18:00:58.013549725 (25h15m10.48948819s) [noise=true , address=77.90.31.13:4626 , piece_id=3IJ3VRZXETGDTMEJD2GP62ZOSRB52CKU3LTIJVJT5RY4TUL4CKRQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.071951296 (25h15m10.547889761s) [noise=true , address=77.90.31.17:4772 , piece_id=MSRKPWDE4SRVTMMJYJPY5SIABTXWU3GCF6DUDRAKCEMCC4P64HKA, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.091470572 (25h15m10.567409037s) [noise=true , address=45.91.116.6:4974 , piece_id=G53ZF3N5QARPXZBMFYBEGN3BIWIS72C5TTFNPGC4ROYSI2O72E4Q, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.123270736 (25h15m10.599209201s) [noise=true , address=104.36.82.7:4911 , piece_id=SU7LULMMHKTQEDXW7JOFYNHZLZCRWAWM63WBNNZ5QKQI4JXEXEYQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.135531534 (25h15m10.611470009s) [noise=true , address=45.91.116.10:4083 , piece_id=LT7Y4QUQBSGYGQB5G5BRU55MWEQRB5KCPYHSUXQCRSHOYJIUKRRA, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.136228077 (25h15m10.612166542s) [noise=true , address=104.36.82.14:4221 , piece_id=DXWRB2WXQ6YFQH4BORZWJJ2QCRM6X5KHAZPZWSUBXJQ2GWXFEOPA, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.149006847 (25h15m10.624945312s) [noise=true , address=45.91.116.14:4189 , piece_id=RUGRHEVJUBNXHEAJF3QCHX5J7VXCP6HB3YFMAAZ6MCYSUN37Z2SQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.153347341 (25h15m10.629285806s) [noise=true , address=45.91.116.11:4119 , piece_id=6JGY6CEHPGQILT774XMHHOWSAFGTDRBPYWF5ATRCYBZ4O6KH5VGQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.168506256 (25h15m10.644444721s) [noise=true , address=104.36.82.2:4729 , piece_id=7S2VDHNG26CGKFVARA5MGE33D5BDA5PUQ4PBQQV4RIMMVNLUXSGA, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.692137487 (25h15m11.168075952s) [noise=true , address=45.91.116.7:4011 , piece_id=EEHSI72I5G4ICXNQBA5USWETF3IWHXKUMWHJPWT757NHHOLH542Q, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.729414359 (25h15m11.205352824s) [noise=true , address=45.91.116.4:4940 , piece_id=JLNQLKSX4QAAHFFE7Z6WJGHJRQ43OBCNQWVHKUSFQ2LBP6NGI2XQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:58.806030670 (25h15m11.281969135s) [noise=true , address=104.36.82.16:4294 , piece_id=ZLCMQWXUUCTBPOJGY6IJAP3KXG4AQMYIVFGPG36MSSOE2CQE3GGQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:59.304266401 (25h15m11.780204866s) [noise=true , address=77.90.31.10:4517 , piece_id=KSERBGMKK2LYDTGS7LAPCGYTBLBTIR26AG5KKJP4JTVHFNDMTNRQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: <nil>
Jul 29 18:00:59.304298112 (25h15m11.780236577s) [seg_pos={0 0}, upload=24174]: Segment reached optimal threshold of 13 pieces.
Jul 29 18:00:59.304454411 (25h15m11.780392876s) [noise=true , address=45.91.116.8:4045 , piece_id=H3PSQ2E33P5HA5SALHCHHXL3OGXMTINHC4ABU32IDZGZEMLOW34Q, seg_pos={0 0}, upload=24174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF
Jul 29 18:00:59.304544283 (25h15m11.780482748s) [noise=true , address=45.91.116.7:4009 , piece_id=WR7GIDXACT2DDBO3RAZY2F6EEW53ACA4ZREPP2MKZMME6L2K7IRA, seg_pos={0 0}, upload=24174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF
Jul 29 18:00:59.304575292 (25h15m11.780513757s) [noise=true , address=104.36.82.4:4804 , piece_id=TGOL5VOG4KVW7GRA5JDT7R54SAJBBQTYKP6UVGUHWK4NS4YYF2UQ, seg_pos={0 0}, upload=24174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 29 18:00:59.304606281 (25h15m11.780544746s) [seg_pos={0 0}, upload=24174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 18:00:59.304686615 (25h15m11.78062508s) [upload=24174]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 18:00:59.304698428 (25h15m11.780636893s) [upload=24174]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 18:00:59.601530762 (25h15m12.077469217s) [upload=24174]: Done uploading segments...
Jul 29 18:00:59.601549638 (25h15m12.077488103s) [upload=24174]: Upload finished. err: <nil>
Duration: 30.002158365s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....................................X..........X.XX......XX.] duration:
22.763344892s
FAIL (2025-07-29 18:01:11 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 22.76337058s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X...X.................................X.............] duration:
1.914100175s
FAIL (2025-07-29 18:01:05 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.914127308s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....................................X..........X.XX......XX] duration:
30.002506284s
FAIL (2025-07-29 18:00:47 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 30.002521352s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001934059s
FAIL (2025-07-29 18:00:16 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF
Jul 29 17:59:00.865136249 (25h13m13.341074714s) [seg_pos={0 0}, upload=24144]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 17:59:00.865221712 (25h13m13.341160177s) [upload=24144]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 17:59:00.865227162 (25h13m13.341165627s) [upload=24144]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 17:59:01.318995263 (25h13m13.794933728s) [upload=24144]: Done uploading segments...
Jul 29 17:59:01.319018307 (25h13m13.794956772s) [upload=24144]: Upload finished. err: <nil>
Jul 29 17:59:46.808970042 (25h13m59.284908507s) [upload=24155]: Starting upload
Jul 29 17:59:46.809043171 (25h13m59.284981636s) [upload=24155]: Done starting upload
Jul 29 17:59:46.809140768 (25h13m59.285079234s) [upload=24155]: Uploading segments...
Jul 29 17:59:46.809237142 (25h13m59.285175607s) [upload=24155]: Got next segment. Inline: false
Jul 29 17:59:46.809285575 (25h13m59.28522404s) [upload=24155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 29 17:59:46.809291526 (25h13m59.285229991s) [upload=24155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 29 17:59:47.044346801 (25h13m59.520285266s) [seg_pos={0 0}, upload=24155]: Begin upload segment...
Jul 29 17:59:47.044395925 (25h13m59.52033439s) [seg_pos={0 0}, upload=24155]: Done begin upload segment.
Jul 29 17:59:47.044415523 (25h13m59.520353988s) [noise=true , address=77.90.31.12:4592 , piece_id=GL7AXI5STSLGGW4MUQA6ZVY6AUJZPHPPDPHJY7Y2TWYAGJS7CG6Q, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.044437044 (25h13m59.520375509s) [noise=true , address=45.91.116.6:4975 , piece_id=JB55HWZNYGC5LSXNT5BKVU25WFCX3EUNULWRXVXZTFRKSPJMHEGQ, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.044804898 (25h13m59.520743363s) [noise=true , address=104.36.82.6:4874 , piece_id=CUKKITM5CKAQWUKDH66BLGKXN2JEFO6W7LESPDTV55YGF5LIAHNQ, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.044836408 (25h13m59.520774873s) [noise=true , address=45.91.116.11:4118 , piece_id=EYEIRYPVIRXKNMMQBQ6HZAUBIZ6WW5MC5OTY2NK4GRTRXWSCFPCA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.045139408 (25h13m59.521077863s) [noise=true , address=45.91.116.5:4298 , piece_id=NHS66MMCCFEAGRF43V7BEGUED64M3PWNSNGHUMVQ7XRGBVMR6THA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.045386761 (25h13m59.521325226s) [noise=true , address=45.91.116.13:4153 , piece_id=D7JK4EJBJKSGFRBUELCVP323YSAFQOBBL7VWW7DW72OKRINH32ZA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.045662589 (25h13m59.521601054s) [noise=true , address=104.36.82.16:4293 , piece_id=K2WGSB7GJ7DEDLK54MCQAQQNXZLESSBDK6M652KGWQTE3JAG3IRA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.045696994 (25h13m59.521635449s) [noise=true , address=45.91.116.11:4117 , piece_id=VUCB7HPMRI6XF4GI6CKRTRMGMK3LPT5266E6OAGH26PTHJQTM6XQ, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.045944708 (25h13m59.521883173s) [noise=true , address=77.90.31.21:4879 , piece_id=BAI4LDV2C5BKEKTZNJYTKYGGOJP6L35AHOUGV7GCJRE2DNHNLXQA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.046194858 (25h13m59.522133323s) [noise=true , address=45.91.116.5:4297 , piece_id=AX4TCM2L2DGC3TVHYMZ4MZYLPYMC66ZDECADX52GNCTNILBXNYWA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.046417474 (25h13m59.522355939s) [noise=true , address=104.36.82.2:4732 , piece_id=WI2PQ2BNUURPDCCF6EXOYUEGGLZCJZE7UZQLHOUXQCSYXVKKOMGA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.046628146 (25h13m59.522566611s) [noise=true , address=104.36.82.7:4909 , piece_id=R73DIRQKTUPFFXH2PVPDFPPGPOLH5IDHUPAT2KGWVAAT7UDO24JA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.046819803 (25h13m59.522758268s) [noise=true , address=77.90.31.18:4806 , piece_id=2GUJGJO5ZMNPKL4KOOCGEGG3MIBSRRJUSUW2CU5E3BWIIQZ4Q7OA, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.047082866 (25h13m59.523021321s) [noise=true , address=104.36.82.9:4006 , piece_id=ED6X34YWCGIG7LVMISACJGJ6N33BXL7WJF7SMNENODJNMPOG3FSQ, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.047208496 (25h13m59.523146961s) [noise=true , address=77.90.31.19:4842 , piece_id=N2SPGQDME4YUXCJ4XORF3MCL557JDP6N25QJFW2MHJ55A2PQWQLQ, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:47.047431614 (25h13m59.523370079s) [noise=true , address=104.36.82.4:4804 , piece_id=24PS564G4JAHU34G6UFR27G4OOWBL7RKGVHKY237UTNMIG2FD4JQ, seg_pos={0 0}, upload=24155]: Uploading piece...
Jul 29 17:59:49.050215559 (25h14m1.526154024s) [upload=24155]: Next returned nil segment
Jul 29 17:59:49.050234586 (25h14m1.526173041s) [upload=24155]: Waiting for error group managing segments...
Jul 29 17:59:49.451242379 (25h14m1.927180844s) [noise=true , address=45.91.116.6:4975 , piece_id=JB55HWZNYGC5LSXNT5BKVU25WFCX3EUNULWRXVXZTFRKSPJMHEGQ, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:49.465412611 (25h14m1.941351076s) [noise=true , address=77.90.31.18:4806 , piece_id=2GUJGJO5ZMNPKL4KOOCGEGG3MIBSRRJUSUW2CU5E3BWIIQZ4Q7OA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:49.709859294 (25h14m2.185797759s) [noise=true , address=104.36.82.6:4874 , piece_id=CUKKITM5CKAQWUKDH66BLGKXN2JEFO6W7LESPDTV55YGF5LIAHNQ, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:49.721902457 (25h14m2.197840922s) [noise=true , address=45.91.116.11:4118 , piece_id=EYEIRYPVIRXKNMMQBQ6HZAUBIZ6WW5MC5OTY2NK4GRTRXWSCFPCA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:50.122025497 (25h14m2.597964003s) [noise=true , address=45.91.116.5:4298 , piece_id=NHS66MMCCFEAGRF43V7BEGUED64M3PWNSNGHUMVQ7XRGBVMR6THA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:50.964778409 (25h14m3.440716874s) [noise=true , address=77.90.31.21:4879 , piece_id=BAI4LDV2C5BKEKTZNJYTKYGGOJP6L35AHOUGV7GCJRE2DNHNLXQA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:51.300670887 (25h14m3.776609352s) [noise=true , address=104.36.82.16:4293 , piece_id=K2WGSB7GJ7DEDLK54MCQAQQNXZLESSBDK6M652KGWQTE3JAG3IRA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:51.531601951 (25h14m4.007540416s) [noise=true , address=45.91.116.5:4297 , piece_id=AX4TCM2L2DGC3TVHYMZ4MZYLPYMC66ZDECADX52GNCTNILBXNYWA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:52.228442946 (25h14m4.704381411s) [noise=true , address=77.90.31.12:4592 , piece_id=GL7AXI5STSLGGW4MUQA6ZVY6AUJZPHPPDPHJY7Y2TWYAGJS7CG6Q, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:52.313377342 (25h14m4.789315807s) [noise=true , address=45.91.116.11:4117 , piece_id=VUCB7HPMRI6XF4GI6CKRTRMGMK3LPT5266E6OAGH26PTHJQTM6XQ, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:52.341323702 (25h14m4.817262167s) [noise=true , address=45.91.116.13:4153 , piece_id=D7JK4EJBJKSGFRBUELCVP323YSAFQOBBL7VWW7DW72OKRINH32ZA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:52.347704360 (25h14m4.823642825s) [noise=true , address=104.36.82.4:4804 , piece_id=24PS564G4JAHU34G6UFR27G4OOWBL7RKGVHKY237UTNMIG2FD4JQ, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:53.738158988 (25h14m6.214097453s) [noise=true , address=104.36.82.9:4006 , piece_id=ED6X34YWCGIG7LVMISACJGJ6N33BXL7WJF7SMNENODJNMPOG3FSQ, seg_pos={0 0}, upload=24155]: Done uploading piece. err: <nil>
Jul 29 17:59:53.738185367 (25h14m6.214123832s) [seg_pos={0 0}, upload=24155]: Segment reached optimal threshold of 13 pieces.
Jul 29 17:59:53.738310267 (25h14m6.214248732s) [noise=true , address=77.90.31.19:4842 , piece_id=N2SPGQDME4YUXCJ4XORF3MCL557JDP6N25QJFW2MHJ55A2PQWQLQ, seg_pos={0 0}, upload=24155]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF
Jul 29 17:59:53.738495361 (25h14m6.214433816s) [noise=true , address=104.36.82.7:4909 , piece_id=R73DIRQKTUPFFXH2PVPDFPPGPOLH5IDHUPAT2KGWVAAT7UDO24JA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:50486->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:50486->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50486->104.36.82.7:4909: use of closed network connection
Jul 29 17:59:53.738754416 (25h14m6.214692881s) [noise=true , address=104.36.82.2:4732 , piece_id=WI2PQ2BNUURPDCCF6EXOYUEGGLZCJZE7UZQLHOUXQCSYXVKKOMGA, seg_pos={0 0}, upload=24155]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: read tcp 188.166.225.170:36902->104.36.82.2:4732: use of closed network connection; context canceled
Jul 29 17:59:53.738814050 (25h14m6.214752535s) [seg_pos={0 0}, upload=24155]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 29 17:59:53.738917608 (25h14m6.214856074s) [upload=24155]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 29 17:59:53.738925573 (25h14m6.214864038s) [upload=24155]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 29 17:59:54.021266822 (25h14m6.497205287s) [upload=24155]: Done uploading segments...
Jul 29 17:59:54.021286999 (25h14m6.497225464s) [upload=24155]: Upload finished. err: <nil>
Duration: 30.001955381s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......................................X..........X.XX......X] duration:
30.001696949s
FAIL (2025-07-29 18:00:16 UTC)
orbiter warning: uplink: context deadline exceeded
Duration: 30.001721676s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X...X.................................X............] duration:
1.912875511s
FAIL (2025-07-29 17:59:51 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.912902795s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.......................................X..........X.XX......] duration:
5.962561144s
FAIL (2025-07-29 17:59:29 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 5.962587485s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..........X...X.................................X...........] duration:
1.883233809s
FAIL (2025-07-29 17:58:45 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.88326015s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................X..........X.XX.....] duration:
6.43607365s
FAIL (2025-07-29 17:58:20 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 6.436100059s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........X...X.................................X..........] duration:
2.015046236s
FAIL (2025-07-29 17:58:06 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.015072526s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.........................................X..........X.XX....] duration:
6.106600737s
FAIL (2025-07-29 17:57:35 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 6.10661817s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [X.........................................X..........X.XX...] duration:
6.160917838s
FAIL (2025-07-29 17:57:20 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 6.160942927s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X...X.................................X.........] duration:
2.045608381s
FAIL (2025-07-29 17:56:56 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.045635664s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.X.........................................X..........X.XX..] duration:
5.953937886s
FAIL (2025-07-29 17:56:44 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 5.953963335s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..X.........................................X..........X.XX.] duration:
6.241888373s
FAIL (2025-07-29 17:56:23 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 6.241909092s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............X...X.................................X........] duration:
1.991334919s
FAIL (2025-07-29 17:55:52 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.991365527s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...X.........................................X..........X.XX] duration:
6.182101915s
FAIL (2025-07-29 17:55:41 UTC)
orbiter warning: digest mismatch: after d98803faaac80bad496b50bd8334b8501750a2d7b596a09cb4cad033333230f8 != before 75c766e70b6cf6f9bbfb734f28f15c7ff7f3edff68846ce5654d664bed6e1d27
Duration: 6.18213068s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..............X...X.................................X.......] duration:
2.128824048s
FAIL (2025-07-29 17:55:17 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.128850499s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X.........................................X..........X.X] duration:
4.23325713s
FAIL (2025-07-29 17:54:55 UTC)
orbiter warning: uplink: object not found ("orbiter_us_0/4M")
Duration: 4.233282871s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...............X...X.................................X......] duration:
1.857679234s
FAIL (2025-07-29 17:54:50 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.857706066s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [................X...X.................................X.....] duration:
1.968178002s
FAIL (2025-07-29 17:54:28 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.968211217s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....X.........................................X..........X.] duration:
5.856237254s
FAIL (2025-07-29 17:54:06 UTC)
orbiter warning: uplink: too many requests
Duration: 5.856261902s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.................X...X.................................X....] duration:
1.830003843s
FAIL (2025-07-29 17:53:42 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.830027449s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [......X.........................................X..........X] duration:
852.179948ms
FAIL (2025-07-29 17:53:04 UTC)
orbiter warning: uplink: too many requests
Duration: 852.196631ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..................X...X.................................X...] duration:
1.88202765s
FAIL (2025-07-29 17:52:55 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.882057877s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X.........................................X..........] duration:
5.732993829s
FAIL (2025-07-29 17:52:20 UTC)
orbiter warning: digest mismatch: after 29cf06604c29927c5d6cc06411892b9c723fa062754df9c7e04116c48dc5a99f != before 817e1de748783d5e0c4e699033b79ee84124e9c10e24e855d10fd0affcb61124
Duration: 5.733033844s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...................X...X.................................X..] duration:
2.077957595s
FAIL (2025-07-29 17:52:17 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 2.077986501s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X.........................................X.........] duration:
5.643291991s
FAIL (2025-07-29 17:51:41 UTC)
orbiter warning: digest mismatch: after 29cf06604c29927c5d6cc06411892b9c723fa062754df9c7e04116c48dc5a99f != before 817e1de748783d5e0c4e699033b79ee84124e9c10e24e855d10fd0affcb61124
Duration: 5.643348659s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....................X...X.................................X.] duration:
1.823194643s
FAIL (2025-07-29 17:51:15 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.823213969s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X.........................................X........] duration:
5.707777912s
FAIL (2025-07-29 17:50:36 UTC)
orbiter warning: digest mismatch: after 29cf06604c29927c5d6cc06411892b9c723fa062754df9c7e04116c48dc5a99f != before 817e1de748783d5e0c4e699033b79ee84124e9c10e24e855d10fd0affcb61124
Duration: 5.707805314s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....................X...X.................................X] duration:
1.422816033s
FAIL (2025-07-29 17:50:36 UTC)
orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
Duration: 1.422862221s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X...X.................................] duration:
1.949828814s
FAIL (2025-07-29 17:49:44 UTC)
orbiter warning: uplink: too many requests
Duration: 1.949856747s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X.........................................X.......] duration:
5.946841122s
FAIL (2025-07-29 17:49:43 UTC)
orbiter warning: digest mismatch: after 29cf06604c29927c5d6cc06411892b9c723fa062754df9c7e04116c48dc5a99f != before 817e1de748783d5e0c4e699033b79ee84124e9c10e24e855d10fd0affcb61124
Duration: 5.946870628s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X.........................................X......] duration:
5.688660953s
FAIL (2025-07-29 17:48:56 UTC)
orbiter warning: digest mismatch: after 29cf06604c29927c5d6cc06411892b9c723fa062754df9c7e04116c48dc5a99f != before 817e1de748783d5e0c4e699033b79ee84124e9c10e24e855d10fd0affcb61124
Duration: 5.688685589s
core
uplink
us-select-1
us1-placement-20
|