Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.X..............................] duration: 5.989277089s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:46:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 5.989311054s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 254.022587ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:46:04 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 254.094794ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 255.965727ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:45:59 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 255.993559ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............................X.X.............................] duration: 13.710142719s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:45:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 13.710175139s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 256.781351ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:45:44 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 256.815395ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 247.978298ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:45:43 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 248.006482ms edge gateway vhost bgp eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...........] duration: 3.404045928s trace: https://jaeger.dev.storj.io/trace/6af15a3a67b655d5 FAIL (2025-11-21 11:45:39 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.404071316s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 257.797807ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:45:29 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 257.823736ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.X............................] duration: 7.210163631s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:45:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 7.210202133s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......X...X.X......X.XX...X.X.X....X............X....X.X.X..] duration: 6.252699759s trace: https://jaeger.dev.storj.io/trace/101ea242871ed19f FAIL (2025-11-21 11:45:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, Piece ID: 5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ) context deadline exceeded; piecestore: (Node ID: 12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, Piece ID: 37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A) context deadline exceeded ======LOGS====== segments... Nov 21 11:43:20.204234549 (291h39m18.75570502s) [upload=256848]: Got next segment. Inline: false Nov 21 11:43:20.204471050 (291h39m18.755941511s) [upload=256848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:43:20.204483864 (291h39m18.755954325s) [upload=256848]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:43:20.446777367 (291h39m18.998247828s) [seg_pos={0 0}, upload=256848]: Begin upload segment... Nov 21 11:43:20.446820980 (291h39m18.998291441s) [seg_pos={0 0}, upload=256848]: Done begin upload segment. Nov 21 11:43:20.446848352 (291h39m18.998318803s) [noise=true , address=145.79.227.27:4689 , piece_id=6TUP3PTJFI3YFY44KMX3K47W5TZVCCTOKWGACT3WECMAHCASMKRQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.446987517 (291h39m18.998457978s) [noise=true , address=45.91.116.5:4298 , piece_id=37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447622606 (291h39m18.999093067s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447636402 (291h39m18.999106863s) [noise=true , address=145.79.227.30:4763 , piece_id=PXJUO4NXHUWNGQC6Z2B77BNRCYHNOSCEWIF6MTLUL6CK7KZSMUZA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447939430 (291h39m18.999409891s) [noise=true , address=77.90.31.14:4663 , piece_id=EC4CSCIDKPZ67CRRV6RJMB2IWK7BCNNJIY2MKWOTLSPKN2QLAURQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448097069 (291h39m18.99956754s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448430825 (291h39m18.999901286s) [noise=true , address=145.79.227.20:4439 , piece_id=ZUD4ZGFHA62TD3OGPOORE7QK42M5GOFE4X2SAPCKQCFGGWMMTLCQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448733472 (291h39m19.000203933s) [noise=true , address=145.79.227.23:4546 , piece_id=77YE2SGB6GYSPFE6SUF5HBQHS26TMVTC7H5EBRJ5HP4RRCKAUR3A, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448760492 (291h39m19.000230953s) [noise=true , address=45.91.116.10:4083 , piece_id=R74YI3RS374FIMXFWZLGVNQ5DMV474COMMR7BKATZ3JIYF47EMCQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449064632 (291h39m19.000535093s) [noise=true , address=145.79.227.25:4619 , piece_id=XBGJDPMF6DMW7DZLV7I2ZGWIX2VYYAZJM5FTAAYZT45CSIGQQ6PA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449210499 (291h39m19.00068096s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449487006 (291h39m19.000957467s) [noise=true , address=145.79.227.21:4475 , piece_id=3AOZCPNQELST3QGS7HQBC32TFE2TMT43QJWCLTN2FZKQF5AYRERQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.450006135 (291h39m19.001476596s) [noise=true , address=145.79.227.28:4726 , piece_id=OJSJA4TD6WIDX4YA57WWJGHTO5KHEL6HYX2RUXUKADFJTYYBVKSQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.450773806 (291h39m19.002244267s) [noise=true , address=145.79.227.8:4330 , piece_id=CCDDYEGXBN5Q4XOHU7IFLLGDZOV4FSZXGLINXNHJWJRP57WPKFVA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.451471896 (291h39m19.002942358s) [noise=true , address=77.90.31.15:4699 , piece_id=5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.452670759 (291h39m19.00414122s) [noise=true , address=145.79.227.28:4725 , piece_id=4Y3XL2Y4X2YUINRQQ2EXSI6GZOSVHLIB3UFVEDI55JI5Q2L6P5KQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:22.149896920 (291h39m20.701367371s) [upload=256848]: Next returned nil segment Nov 21 11:43:22.149913270 (291h39m20.701383731s) [upload=256848]: Waiting for error group managing segments... Nov 21 11:43:22.437692958 (291h39m20.989163409s) [noise=true , address=145.79.227.25:4619 , piece_id=XBGJDPMF6DMW7DZLV7I2ZGWIX2VYYAZJM5FTAAYZT45CSIGQQ6PA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.438268674 (291h39m20.989739135s) [noise=true , address=145.79.227.21:4475 , piece_id=3AOZCPNQELST3QGS7HQBC32TFE2TMT43QJWCLTN2FZKQF5AYRERQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.439554944 (291h39m20.991025395s) [noise=true , address=145.79.227.20:4439 , piece_id=ZUD4ZGFHA62TD3OGPOORE7QK42M5GOFE4X2SAPCKQCFGGWMMTLCQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.443711388 (291h39m20.995181849s) [noise=true , address=145.79.227.23:4546 , piece_id=77YE2SGB6GYSPFE6SUF5HBQHS26TMVTC7H5EBRJ5HP4RRCKAUR3A, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.446473247 (291h39m20.997943709s) [noise=true , address=145.79.227.8:4330 , piece_id=CCDDYEGXBN5Q4XOHU7IFLLGDZOV4FSZXGLINXNHJWJRP57WPKFVA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.459544016 (291h39m21.011014477s) [noise=true , address=145.79.227.30:4763 , piece_id=PXJUO4NXHUWNGQC6Z2B77BNRCYHNOSCEWIF6MTLUL6CK7KZSMUZA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.462636585 (291h39m21.014107046s) [noise=true , address=145.79.227.28:4725 , piece_id=4Y3XL2Y4X2YUINRQQ2EXSI6GZOSVHLIB3UFVEDI55JI5Q2L6P5KQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.462753898 (291h39m21.014224359s) [noise=true , address=145.79.227.28:4726 , piece_id=OJSJA4TD6WIDX4YA57WWJGHTO5KHEL6HYX2RUXUKADFJTYYBVKSQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.471210822 (291h39m21.022681284s) [noise=true , address=145.79.227.27:4689 , piece_id=6TUP3PTJFI3YFY44KMX3K47W5TZVCCTOKWGACT3WECMAHCASMKRQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.044445199 (291h39m21.595915651s) [noise=true , address=45.91.116.5:4298 , piece_id=37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.050576524 (291h39m21.602046975s) [noise=true , address=77.90.31.14:4663 , piece_id=EC4CSCIDKPZ67CRRV6RJMB2IWK7BCNNJIY2MKWOTLSPKN2QLAURQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.071209674 (291h39m21.622680135s) [noise=true , address=45.91.116.10:4083 , piece_id=R74YI3RS374FIMXFWZLGVNQ5DMV474COMMR7BKATZ3JIYF47EMCQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.085286689 (291h39m21.636757151s) [noise=true , address=77.90.31.15:4699 , piece_id=5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.085313470 (291h39m21.636783931s) [seg_pos={0 0}, upload=256848]: Segment reached optimal threshold of 13 pieces. Nov 21 11:43:23.085591740 (291h39m21.637062201s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection Nov 21 11:43:23.085601990 (291h39m21.637072451s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.085776041 (291h39m21.637246512s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Nov 21 11:43:23.085879939 (291h39m21.63735041s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.086018453 (291h39m21.637488924s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection Nov 21 11:43:23.086029934 (291h39m21.637500395s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.086057527 (291h39m21.637527988s) [seg_pos={0 0}, upload=256848]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:43:23.086152668 (291h39m21.637623119s) [upload=256848]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:43:23.086161144 (291h39m21.637631606s) [upload=256848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:43:23.379337949 (291h39m21.93080841s) [upload=256848]: Done uploading segments... Nov 21 11:43:23.379374438 (291h39m21.930844899s) [upload=256848]: Upload finished. err: <nil> Duration: 6.252723723s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......X...X.X......X.XX...X.X.X....X............X....X.X.X.] duration: 28.783625526s trace: https://jaeger.dev.storj.io/trace/101ea242871ed19f FAIL (2025-11-21 11:44:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, Piece ID: 5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ) context deadline exceeded; piecestore: (Node ID: 12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, Piece ID: 37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A) context deadline exceeded ======LOGS====== segments... Nov 21 11:43:20.204234549 (291h39m18.75570502s) [upload=256848]: Got next segment. Inline: false Nov 21 11:43:20.204471050 (291h39m18.755941511s) [upload=256848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:43:20.204483864 (291h39m18.755954325s) [upload=256848]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:43:20.446777367 (291h39m18.998247828s) [seg_pos={0 0}, upload=256848]: Begin upload segment... Nov 21 11:43:20.446820980 (291h39m18.998291441s) [seg_pos={0 0}, upload=256848]: Done begin upload segment. Nov 21 11:43:20.446848352 (291h39m18.998318803s) [noise=true , address=145.79.227.27:4689 , piece_id=6TUP3PTJFI3YFY44KMX3K47W5TZVCCTOKWGACT3WECMAHCASMKRQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.446987517 (291h39m18.998457978s) [noise=true , address=45.91.116.5:4298 , piece_id=37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447622606 (291h39m18.999093067s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447636402 (291h39m18.999106863s) [noise=true , address=145.79.227.30:4763 , piece_id=PXJUO4NXHUWNGQC6Z2B77BNRCYHNOSCEWIF6MTLUL6CK7KZSMUZA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447939430 (291h39m18.999409891s) [noise=true , address=77.90.31.14:4663 , piece_id=EC4CSCIDKPZ67CRRV6RJMB2IWK7BCNNJIY2MKWOTLSPKN2QLAURQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448097069 (291h39m18.99956754s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448430825 (291h39m18.999901286s) [noise=true , address=145.79.227.20:4439 , piece_id=ZUD4ZGFHA62TD3OGPOORE7QK42M5GOFE4X2SAPCKQCFGGWMMTLCQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448733472 (291h39m19.000203933s) [noise=true , address=145.79.227.23:4546 , piece_id=77YE2SGB6GYSPFE6SUF5HBQHS26TMVTC7H5EBRJ5HP4RRCKAUR3A, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448760492 (291h39m19.000230953s) [noise=true , address=45.91.116.10:4083 , piece_id=R74YI3RS374FIMXFWZLGVNQ5DMV474COMMR7BKATZ3JIYF47EMCQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449064632 (291h39m19.000535093s) [noise=true , address=145.79.227.25:4619 , piece_id=XBGJDPMF6DMW7DZLV7I2ZGWIX2VYYAZJM5FTAAYZT45CSIGQQ6PA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449210499 (291h39m19.00068096s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449487006 (291h39m19.000957467s) [noise=true , address=145.79.227.21:4475 , piece_id=3AOZCPNQELST3QGS7HQBC32TFE2TMT43QJWCLTN2FZKQF5AYRERQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.450006135 (291h39m19.001476596s) [noise=true , address=145.79.227.28:4726 , piece_id=OJSJA4TD6WIDX4YA57WWJGHTO5KHEL6HYX2RUXUKADFJTYYBVKSQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.450773806 (291h39m19.002244267s) [noise=true , address=145.79.227.8:4330 , piece_id=CCDDYEGXBN5Q4XOHU7IFLLGDZOV4FSZXGLINXNHJWJRP57WPKFVA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.451471896 (291h39m19.002942358s) [noise=true , address=77.90.31.15:4699 , piece_id=5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.452670759 (291h39m19.00414122s) [noise=true , address=145.79.227.28:4725 , piece_id=4Y3XL2Y4X2YUINRQQ2EXSI6GZOSVHLIB3UFVEDI55JI5Q2L6P5KQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:22.149896920 (291h39m20.701367371s) [upload=256848]: Next returned nil segment Nov 21 11:43:22.149913270 (291h39m20.701383731s) [upload=256848]: Waiting for error group managing segments... Nov 21 11:43:22.437692958 (291h39m20.989163409s) [noise=true , address=145.79.227.25:4619 , piece_id=XBGJDPMF6DMW7DZLV7I2ZGWIX2VYYAZJM5FTAAYZT45CSIGQQ6PA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.438268674 (291h39m20.989739135s) [noise=true , address=145.79.227.21:4475 , piece_id=3AOZCPNQELST3QGS7HQBC32TFE2TMT43QJWCLTN2FZKQF5AYRERQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.439554944 (291h39m20.991025395s) [noise=true , address=145.79.227.20:4439 , piece_id=ZUD4ZGFHA62TD3OGPOORE7QK42M5GOFE4X2SAPCKQCFGGWMMTLCQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.443711388 (291h39m20.995181849s) [noise=true , address=145.79.227.23:4546 , piece_id=77YE2SGB6GYSPFE6SUF5HBQHS26TMVTC7H5EBRJ5HP4RRCKAUR3A, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.446473247 (291h39m20.997943709s) [noise=true , address=145.79.227.8:4330 , piece_id=CCDDYEGXBN5Q4XOHU7IFLLGDZOV4FSZXGLINXNHJWJRP57WPKFVA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.459544016 (291h39m21.011014477s) [noise=true , address=145.79.227.30:4763 , piece_id=PXJUO4NXHUWNGQC6Z2B77BNRCYHNOSCEWIF6MTLUL6CK7KZSMUZA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.462636585 (291h39m21.014107046s) [noise=true , address=145.79.227.28:4725 , piece_id=4Y3XL2Y4X2YUINRQQ2EXSI6GZOSVHLIB3UFVEDI55JI5Q2L6P5KQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.462753898 (291h39m21.014224359s) [noise=true , address=145.79.227.28:4726 , piece_id=OJSJA4TD6WIDX4YA57WWJGHTO5KHEL6HYX2RUXUKADFJTYYBVKSQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.471210822 (291h39m21.022681284s) [noise=true , address=145.79.227.27:4689 , piece_id=6TUP3PTJFI3YFY44KMX3K47W5TZVCCTOKWGACT3WECMAHCASMKRQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.044445199 (291h39m21.595915651s) [noise=true , address=45.91.116.5:4298 , piece_id=37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.050576524 (291h39m21.602046975s) [noise=true , address=77.90.31.14:4663 , piece_id=EC4CSCIDKPZ67CRRV6RJMB2IWK7BCNNJIY2MKWOTLSPKN2QLAURQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.071209674 (291h39m21.622680135s) [noise=true , address=45.91.116.10:4083 , piece_id=R74YI3RS374FIMXFWZLGVNQ5DMV474COMMR7BKATZ3JIYF47EMCQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.085286689 (291h39m21.636757151s) [noise=true , address=77.90.31.15:4699 , piece_id=5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.085313470 (291h39m21.636783931s) [seg_pos={0 0}, upload=256848]: Segment reached optimal threshold of 13 pieces. Nov 21 11:43:23.085591740 (291h39m21.637062201s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection Nov 21 11:43:23.085601990 (291h39m21.637072451s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.085776041 (291h39m21.637246512s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Nov 21 11:43:23.085879939 (291h39m21.63735041s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.086018453 (291h39m21.637488924s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection Nov 21 11:43:23.086029934 (291h39m21.637500395s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.086057527 (291h39m21.637527988s) [seg_pos={0 0}, upload=256848]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:43:23.086152668 (291h39m21.637623119s) [upload=256848]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:43:23.086161144 (291h39m21.637631606s) [upload=256848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:43:23.379337949 (291h39m21.93080841s) [upload=256848]: Done uploading segments... Nov 21 11:43:23.379374438 (291h39m21.930844899s) [upload=256848]: Upload finished. err: <nil> Duration: 28.78364886s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..........] duration: 3.387847033s trace: https://jaeger.dev.storj.io/trace/6af15a3a67b655d5 FAIL (2025-11-21 11:44:53 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.387865227s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 258.527264ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:44:52 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 258.547965ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 254.627349ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:44:38 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 254.672345ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 249.722379ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:44:36 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 249.786762ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 258.27249ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:44:33 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 258.300053ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 280.107779ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:44:32 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 280.135332ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 261.939663ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:44:15 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 261.968047ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.X...........................] duration: 5.774116665s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:43:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 5.774143497s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 260.071555ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:43:51 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 260.126019ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........X...X.X......X.XX...X.X.X....X............X....X.X.X] duration: 30.003348891s trace: https://jaeger.dev.storj.io/trace/101ea242871ed19f FAIL (2025-11-21 11:43:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, Piece ID: 5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ) context deadline exceeded; piecestore: (Node ID: 12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, Piece ID: 37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A) context deadline exceeded ======LOGS====== segments... Nov 21 11:43:20.204234549 (291h39m18.75570502s) [upload=256848]: Got next segment. Inline: false Nov 21 11:43:20.204471050 (291h39m18.755941511s) [upload=256848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:43:20.204483864 (291h39m18.755954325s) [upload=256848]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:43:20.446777367 (291h39m18.998247828s) [seg_pos={0 0}, upload=256848]: Begin upload segment... Nov 21 11:43:20.446820980 (291h39m18.998291441s) [seg_pos={0 0}, upload=256848]: Done begin upload segment. Nov 21 11:43:20.446848352 (291h39m18.998318803s) [noise=true , address=145.79.227.27:4689 , piece_id=6TUP3PTJFI3YFY44KMX3K47W5TZVCCTOKWGACT3WECMAHCASMKRQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.446987517 (291h39m18.998457978s) [noise=true , address=45.91.116.5:4298 , piece_id=37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447622606 (291h39m18.999093067s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447636402 (291h39m18.999106863s) [noise=true , address=145.79.227.30:4763 , piece_id=PXJUO4NXHUWNGQC6Z2B77BNRCYHNOSCEWIF6MTLUL6CK7KZSMUZA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.447939430 (291h39m18.999409891s) [noise=true , address=77.90.31.14:4663 , piece_id=EC4CSCIDKPZ67CRRV6RJMB2IWK7BCNNJIY2MKWOTLSPKN2QLAURQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448097069 (291h39m18.99956754s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448430825 (291h39m18.999901286s) [noise=true , address=145.79.227.20:4439 , piece_id=ZUD4ZGFHA62TD3OGPOORE7QK42M5GOFE4X2SAPCKQCFGGWMMTLCQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448733472 (291h39m19.000203933s) [noise=true , address=145.79.227.23:4546 , piece_id=77YE2SGB6GYSPFE6SUF5HBQHS26TMVTC7H5EBRJ5HP4RRCKAUR3A, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.448760492 (291h39m19.000230953s) [noise=true , address=45.91.116.10:4083 , piece_id=R74YI3RS374FIMXFWZLGVNQ5DMV474COMMR7BKATZ3JIYF47EMCQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449064632 (291h39m19.000535093s) [noise=true , address=145.79.227.25:4619 , piece_id=XBGJDPMF6DMW7DZLV7I2ZGWIX2VYYAZJM5FTAAYZT45CSIGQQ6PA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449210499 (291h39m19.00068096s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.449487006 (291h39m19.000957467s) [noise=true , address=145.79.227.21:4475 , piece_id=3AOZCPNQELST3QGS7HQBC32TFE2TMT43QJWCLTN2FZKQF5AYRERQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.450006135 (291h39m19.001476596s) [noise=true , address=145.79.227.28:4726 , piece_id=OJSJA4TD6WIDX4YA57WWJGHTO5KHEL6HYX2RUXUKADFJTYYBVKSQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.450773806 (291h39m19.002244267s) [noise=true , address=145.79.227.8:4330 , piece_id=CCDDYEGXBN5Q4XOHU7IFLLGDZOV4FSZXGLINXNHJWJRP57WPKFVA, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.451471896 (291h39m19.002942358s) [noise=true , address=77.90.31.15:4699 , piece_id=5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:20.452670759 (291h39m19.00414122s) [noise=true , address=145.79.227.28:4725 , piece_id=4Y3XL2Y4X2YUINRQQ2EXSI6GZOSVHLIB3UFVEDI55JI5Q2L6P5KQ, seg_pos={0 0}, upload=256848]: Uploading piece... Nov 21 11:43:22.149896920 (291h39m20.701367371s) [upload=256848]: Next returned nil segment Nov 21 11:43:22.149913270 (291h39m20.701383731s) [upload=256848]: Waiting for error group managing segments... Nov 21 11:43:22.437692958 (291h39m20.989163409s) [noise=true , address=145.79.227.25:4619 , piece_id=XBGJDPMF6DMW7DZLV7I2ZGWIX2VYYAZJM5FTAAYZT45CSIGQQ6PA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.438268674 (291h39m20.989739135s) [noise=true , address=145.79.227.21:4475 , piece_id=3AOZCPNQELST3QGS7HQBC32TFE2TMT43QJWCLTN2FZKQF5AYRERQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.439554944 (291h39m20.991025395s) [noise=true , address=145.79.227.20:4439 , piece_id=ZUD4ZGFHA62TD3OGPOORE7QK42M5GOFE4X2SAPCKQCFGGWMMTLCQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.443711388 (291h39m20.995181849s) [noise=true , address=145.79.227.23:4546 , piece_id=77YE2SGB6GYSPFE6SUF5HBQHS26TMVTC7H5EBRJ5HP4RRCKAUR3A, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.446473247 (291h39m20.997943709s) [noise=true , address=145.79.227.8:4330 , piece_id=CCDDYEGXBN5Q4XOHU7IFLLGDZOV4FSZXGLINXNHJWJRP57WPKFVA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.459544016 (291h39m21.011014477s) [noise=true , address=145.79.227.30:4763 , piece_id=PXJUO4NXHUWNGQC6Z2B77BNRCYHNOSCEWIF6MTLUL6CK7KZSMUZA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.462636585 (291h39m21.014107046s) [noise=true , address=145.79.227.28:4725 , piece_id=4Y3XL2Y4X2YUINRQQ2EXSI6GZOSVHLIB3UFVEDI55JI5Q2L6P5KQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.462753898 (291h39m21.014224359s) [noise=true , address=145.79.227.28:4726 , piece_id=OJSJA4TD6WIDX4YA57WWJGHTO5KHEL6HYX2RUXUKADFJTYYBVKSQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:22.471210822 (291h39m21.022681284s) [noise=true , address=145.79.227.27:4689 , piece_id=6TUP3PTJFI3YFY44KMX3K47W5TZVCCTOKWGACT3WECMAHCASMKRQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.044445199 (291h39m21.595915651s) [noise=true , address=45.91.116.5:4298 , piece_id=37LV4J4WIUQ6JFEEVUP4R7ESINP3SUU3GDWU3SEIRNWKBDQ6LX4A, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.050576524 (291h39m21.602046975s) [noise=true , address=77.90.31.14:4663 , piece_id=EC4CSCIDKPZ67CRRV6RJMB2IWK7BCNNJIY2MKWOTLSPKN2QLAURQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.071209674 (291h39m21.622680135s) [noise=true , address=45.91.116.10:4083 , piece_id=R74YI3RS374FIMXFWZLGVNQ5DMV474COMMR7BKATZ3JIYF47EMCQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.085286689 (291h39m21.636757151s) [noise=true , address=77.90.31.15:4699 , piece_id=5YNF5AIUAJ2LSJCBY3KGEZTYMFYMF36OZWPRKFNVDNLBJUJZWTRQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: <nil> Nov 21 11:43:23.085313470 (291h39m21.636783931s) [seg_pos={0 0}, upload=256848]: Segment reached optimal threshold of 13 pieces. Nov 21 11:43:23.085591740 (291h39m21.637062201s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52312->45.91.116.4:4937: use of closed network connection Nov 21 11:43:23.085601990 (291h39m21.637072451s) [noise=true , address=45.91.116.4:4937 , piece_id=XIAAMPS6YHZXAG2MXCE5GZL74ZFL5M4R6VUABKA6PX6Y2K7FHAUQ, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.085776041 (291h39m21.637246512s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Nov 21 11:43:23.085879939 (291h39m21.63735041s) [noise=true , address=77.90.31.12:4592 , piece_id=PQKY7DATMT7VPL56AS7XW3RK6RTFMHGUXNMTQAOI5WN6L7W42GOA, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.086018453 (291h39m21.637488924s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41002->45.91.116.8:4048: use of closed network connection Nov 21 11:43:23.086029934 (291h39m21.637500395s) [noise=true , address=45.91.116.8:4048 , piece_id=SASBGDMZZEU4M5PW4CFN7HTBTLBVKBGUWK35VXYFELXCID5XCXGA, seg_pos={0 0}, upload=256848]: Long tail cancellation due to optimal threshold Nov 21 11:43:23.086057527 (291h39m21.637527988s) [seg_pos={0 0}, upload=256848]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:43:23.086152668 (291h39m21.637623119s) [upload=256848]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:43:23.086161144 (291h39m21.637631606s) [upload=256848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:43:23.379337949 (291h39m21.93080841s) [upload=256848]: Done uploading segments... Nov 21 11:43:23.379374438 (291h39m21.930844899s) [upload=256848]: Upload finished. err: <nil> Duration: 30.00337953s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.........] duration: 3.736434879s trace: https://jaeger.dev.storj.io/trace/6af15a3a67b655d5 FAIL (2025-11-21 11:43:46 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.736461842s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 245.337679ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:43:26 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 245.365842ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 246.92953ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:43:24 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 246.946381ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 256.455489ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:43:21 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 256.484355ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.X..........................] duration: 5.872980569s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:43:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 5.873011118s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 263.812478ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:43:13 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 263.844489ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 247.419654ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:43:07 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 247.445624ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........X...X.X......X.XX...X.X.X....X............X....X.X.] duration: 6.335925573s trace: https://jaeger.dev.storj.io/trace/4f6e48f588fcce83 FAIL (2025-11-21 11:43:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 2N4UQ5UNTKSQQMFZEG2UFSNE5AISQ7I4O5JJYX7TPZN32BUG4TDA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: FRCQYZVW5GX7YURDKYLDBLOSRJPY7MYOVL5W7LVJ3QV75L53FAEQ) context deadline exceeded ======LOGS====== 1 11:42:27.582346605 (291h38m26.133817066s) [upload=256832]: Starting upload Nov 21 11:42:27.582419664 (291h38m26.133890125s) [upload=256832]: Done starting upload Nov 21 11:42:27.582484858 (291h38m26.133955319s) [upload=256832]: Uploading segments... Nov 21 11:42:27.582628000 (291h38m26.134098451s) [upload=256832]: Got next segment. Inline: false Nov 21 11:42:27.582668327 (291h38m26.134138788s) [upload=256832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:42:27.582672956 (291h38m26.134143407s) [upload=256832]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:42:27.827064344 (291h38m26.378534806s) [seg_pos={0 0}, upload=256832]: Begin upload segment... Nov 21 11:42:27.827119079 (291h38m26.37858954s) [seg_pos={0 0}, upload=256832]: Done begin upload segment. Nov 21 11:42:27.827153535 (291h38m26.378623996s) [noise=true , address=77.90.31.17:4770 , piece_id=RWJFMY524OVK57SNMQ7YFMPQRID6QS2SHUX57KE6PBIRDTVRI6XQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.827191507 (291h38m26.378661958s) [noise=true , address=145.79.227.22:4509 , piece_id=XFR3JCYUNAA6XNRALD2FKC3MAWUCP7SZTLO64YFQO3DTLNFBFQFA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.827949740 (291h38m26.379420201s) [noise=true , address=45.91.116.8:4048 , piece_id=2N4UQ5UNTKSQQMFZEG2UFSNE5AISQ7I4O5JJYX7TPZN32BUG4TDA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.828041796 (291h38m26.379512257s) [noise=true , address=145.79.227.30:4762 , piece_id=X4U5XOY3WCTLSFVMPO46IYXWLS7CXBKXVYJTR5M3EYUSN4H4FMIQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.828655685 (291h38m26.380126136s) [noise=true , address=45.91.116.4:4940 , piece_id=DTQHJJGXOXE6B2VHK5JEMAVKOSWNDRTG3CDGSBKMQI3AOV3UWPMQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.828722051 (291h38m26.380192512s) [noise=true , address=145.79.227.20:4439 , piece_id=D4TGIE7NGGWLO3Z4PUTS45BETLZYMWKWGZUZWV27N6YSPZVUUM7A, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.829134015 (291h38m26.380604476s) [noise=true , address=145.79.227.20:4437 , piece_id=6NLNBHESW2YOFY6NPNLIY4OJYSGM4TYO6DD6Z3A5P2CSFV4EH2RQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.829260827 (291h38m26.380731288s) [noise=true , address=145.79.227.25:4617 , piece_id=BGWXWPS6PRMX7QY4526E6ETXZ2ADNFJ6R3N67YXYUPMZU72E2BTA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.829623769 (291h38m26.38109423s) [noise=true , address=77.90.31.15:4697 , piece_id=ZXQUVQZSN27ML5WQ2GOO3BB5G2U75CLEVNSSBG6MZRNH4D6IRVXA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.830286991 (291h38m26.381757483s) [noise=true , address=45.91.116.14:4190 , piece_id=FRCQYZVW5GX7YURDKYLDBLOSRJPY7MYOVL5W7LVJ3QV75L53FAEQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.830564760 (291h38m26.382035221s) [noise=true , address=145.79.227.31:4798 , piece_id=DDWZBNIJSDLNGZXNUSO644NYGG5SNMV7NE4K6L56DPPT3XTAWOZQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.830886302 (291h38m26.382356763s) [noise=true , address=145.79.227.14:4839 , piece_id=FG26V3C4G3RKOTNPI4HBWY3HD6YQNGF2W5PJVKC5OQPTBDYGZYYQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.831939898 (291h38m26.383410359s) [noise=true , address=145.79.227.11:4730 , piece_id=NF243ZYVKX557RFPWNTY6LY3AR3VCQJ3BMFQIUOIV6YRAGEJJTAA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.832166560 (291h38m26.383637021s) [noise=true , address=77.90.31.11:4554 , piece_id=VUDSHBH7QY5IQ57UP4SRKYZ5QCHRRYWU3E4ZDAZTF65LU3ME2WOA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.833257047 (291h38m26.384727508s) [noise=true , address=145.79.227.27:4689 , piece_id=6INJ622RXLRR3JMMWVFEJKEOCQKNUBM4CCHID7V7FIRDPJ2EHUPA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.834108598 (291h38m26.385579059s) [noise=true , address=145.79.227.28:4726 , piece_id=VNCH5X2XDC7U7I2EICAU66EPJ74B4LZGXQDAFKE4HREPMCL7XHTA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:29.443724882 (291h38m27.995195343s) [upload=256832]: Next returned nil segment Nov 21 11:42:29.443741404 (291h38m27.995211875s) [upload=256832]: Waiting for error group managing segments... Nov 21 11:42:29.825233406 (291h38m28.376703857s) [noise=true , address=145.79.227.30:4762 , piece_id=X4U5XOY3WCTLSFVMPO46IYXWLS7CXBKXVYJTR5M3EYUSN4H4FMIQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.828134501 (291h38m28.379604962s) [noise=true , address=145.79.227.31:4798 , piece_id=DDWZBNIJSDLNGZXNUSO644NYGG5SNMV7NE4K6L56DPPT3XTAWOZQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.828916299 (291h38m28.38038676s) [noise=true , address=145.79.227.25:4617 , piece_id=BGWXWPS6PRMX7QY4526E6ETXZ2ADNFJ6R3N67YXYUPMZU72E2BTA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.830188623 (291h38m28.381659084s) [noise=true , address=145.79.227.20:4439 , piece_id=D4TGIE7NGGWLO3Z4PUTS45BETLZYMWKWGZUZWV27N6YSPZVUUM7A, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.831138010 (291h38m28.382608471s) [noise=true , address=145.79.227.11:4730 , piece_id=NF243ZYVKX557RFPWNTY6LY3AR3VCQJ3BMFQIUOIV6YRAGEJJTAA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.831608286 (291h38m28.383078737s) [noise=true , address=145.79.227.20:4437 , piece_id=6NLNBHESW2YOFY6NPNLIY4OJYSGM4TYO6DD6Z3A5P2CSFV4EH2RQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.839411715 (291h38m28.390882166s) [noise=true , address=145.79.227.14:4839 , piece_id=FG26V3C4G3RKOTNPI4HBWY3HD6YQNGF2W5PJVKC5OQPTBDYGZYYQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.839834861 (291h38m28.391305332s) [noise=true , address=145.79.227.22:4509 , piece_id=XFR3JCYUNAA6XNRALD2FKC3MAWUCP7SZTLO64YFQO3DTLNFBFQFA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.846124898 (291h38m28.397595359s) [noise=true , address=145.79.227.27:4689 , piece_id=6INJ622RXLRR3JMMWVFEJKEOCQKNUBM4CCHID7V7FIRDPJ2EHUPA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.854921609 (291h38m28.40639206s) [noise=true , address=145.79.227.28:4726 , piece_id=VNCH5X2XDC7U7I2EICAU66EPJ74B4LZGXQDAFKE4HREPMCL7XHTA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.404759674 (291h38m28.956230146s) [noise=true , address=45.91.116.8:4048 , piece_id=2N4UQ5UNTKSQQMFZEG2UFSNE5AISQ7I4O5JJYX7TPZN32BUG4TDA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.531857242 (291h38m29.083327693s) [noise=true , address=45.91.116.14:4190 , piece_id=FRCQYZVW5GX7YURDKYLDBLOSRJPY7MYOVL5W7LVJ3QV75L53FAEQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.758748116 (291h38m29.310218577s) [noise=true , address=77.90.31.11:4554 , piece_id=VUDSHBH7QY5IQ57UP4SRKYZ5QCHRRYWU3E4ZDAZTF65LU3ME2WOA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.758774556 (291h38m29.310245017s) [seg_pos={0 0}, upload=256832]: Segment reached optimal threshold of 13 pieces. Nov 21 11:42:30.759205336 (291h38m29.310675797s) [noise=true , address=77.90.31.17:4770 , piece_id=RWJFMY524OVK57SNMQ7YFMPQRID6QS2SHUX57KE6PBIRDTVRI6XQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Nov 21 11:42:30.759245923 (291h38m29.310716374s) [noise=true , address=77.90.31.17:4770 , piece_id=RWJFMY524OVK57SNMQ7YFMPQRID6QS2SHUX57KE6PBIRDTVRI6XQ, seg_pos={0 0}, upload=256832]: Long tail cancellation due to optimal threshold Nov 21 11:42:30.759290920 (291h38m29.310761381s) [noise=true , address=77.90.31.15:4697 , piece_id=ZXQUVQZSN27ML5WQ2GOO3BB5G2U75CLEVNSSBG6MZRNH4D6IRVXA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Nov 21 11:42:30.759301820 (291h38m29.310772281s) [noise=true , address=77.90.31.15:4697 , piece_id=ZXQUVQZSN27ML5WQ2GOO3BB5G2U75CLEVNSSBG6MZRNH4D6IRVXA, seg_pos={0 0}, upload=256832]: Long tail cancellation due to optimal threshold Nov 21 11:42:30.759587844 (291h38m29.311058306s) [noise=true , address=45.91.116.4:4940 , piece_id=DTQHJJGXOXE6B2VHK5JEMAVKOSWNDRTG3CDGSBKMQI3AOV3UWPMQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: write tcp 188.166.225.170:56388->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56388->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56388->45.91.116.4:4940: use of closed network connection Nov 21 11:42:30.759598234 (291h38m29.311068695s) [noise=true , address=45.91.116.4:4940 , piece_id=DTQHJJGXOXE6B2VHK5JEMAVKOSWNDRTG3CDGSBKMQI3AOV3UWPMQ, seg_pos={0 0}, upload=256832]: Long tail cancellation due to optimal threshold Nov 21 11:42:30.759624644 (291h38m29.311095106s) [seg_pos={0 0}, upload=256832]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:42:30.759725727 (291h38m29.311196188s) [upload=256832]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:42:30.759734524 (291h38m29.311204985s) [upload=256832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:42:31.046104166 (291h38m29.597574617s) [upload=256832]: Done uploading segments... Nov 21 11:42:31.046167967 (291h38m29.597638418s) [upload=256832]: Upload finished. err: <nil> Duration: 6.335947474s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 256.983375ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:43:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 257.01716ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [................................X.X.........................] duration: 8.338980199s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:43:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 8.339009223s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..........X...X.X......X.XX...X.X.X....X............X....X.X] duration: 30.00342362s trace: https://jaeger.dev.storj.io/trace/4f6e48f588fcce83 FAIL (2025-11-21 11:42:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, Piece ID: 2N4UQ5UNTKSQQMFZEG2UFSNE5AISQ7I4O5JJYX7TPZN32BUG4TDA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: FRCQYZVW5GX7YURDKYLDBLOSRJPY7MYOVL5W7LVJ3QV75L53FAEQ) context deadline exceeded ======LOGS====== 1 11:42:27.582346605 (291h38m26.133817066s) [upload=256832]: Starting upload Nov 21 11:42:27.582419664 (291h38m26.133890125s) [upload=256832]: Done starting upload Nov 21 11:42:27.582484858 (291h38m26.133955319s) [upload=256832]: Uploading segments... Nov 21 11:42:27.582628000 (291h38m26.134098451s) [upload=256832]: Got next segment. Inline: false Nov 21 11:42:27.582668327 (291h38m26.134138788s) [upload=256832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:42:27.582672956 (291h38m26.134143407s) [upload=256832]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:42:27.827064344 (291h38m26.378534806s) [seg_pos={0 0}, upload=256832]: Begin upload segment... Nov 21 11:42:27.827119079 (291h38m26.37858954s) [seg_pos={0 0}, upload=256832]: Done begin upload segment. Nov 21 11:42:27.827153535 (291h38m26.378623996s) [noise=true , address=77.90.31.17:4770 , piece_id=RWJFMY524OVK57SNMQ7YFMPQRID6QS2SHUX57KE6PBIRDTVRI6XQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.827191507 (291h38m26.378661958s) [noise=true , address=145.79.227.22:4509 , piece_id=XFR3JCYUNAA6XNRALD2FKC3MAWUCP7SZTLO64YFQO3DTLNFBFQFA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.827949740 (291h38m26.379420201s) [noise=true , address=45.91.116.8:4048 , piece_id=2N4UQ5UNTKSQQMFZEG2UFSNE5AISQ7I4O5JJYX7TPZN32BUG4TDA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.828041796 (291h38m26.379512257s) [noise=true , address=145.79.227.30:4762 , piece_id=X4U5XOY3WCTLSFVMPO46IYXWLS7CXBKXVYJTR5M3EYUSN4H4FMIQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.828655685 (291h38m26.380126136s) [noise=true , address=45.91.116.4:4940 , piece_id=DTQHJJGXOXE6B2VHK5JEMAVKOSWNDRTG3CDGSBKMQI3AOV3UWPMQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.828722051 (291h38m26.380192512s) [noise=true , address=145.79.227.20:4439 , piece_id=D4TGIE7NGGWLO3Z4PUTS45BETLZYMWKWGZUZWV27N6YSPZVUUM7A, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.829134015 (291h38m26.380604476s) [noise=true , address=145.79.227.20:4437 , piece_id=6NLNBHESW2YOFY6NPNLIY4OJYSGM4TYO6DD6Z3A5P2CSFV4EH2RQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.829260827 (291h38m26.380731288s) [noise=true , address=145.79.227.25:4617 , piece_id=BGWXWPS6PRMX7QY4526E6ETXZ2ADNFJ6R3N67YXYUPMZU72E2BTA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.829623769 (291h38m26.38109423s) [noise=true , address=77.90.31.15:4697 , piece_id=ZXQUVQZSN27ML5WQ2GOO3BB5G2U75CLEVNSSBG6MZRNH4D6IRVXA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.830286991 (291h38m26.381757483s) [noise=true , address=45.91.116.14:4190 , piece_id=FRCQYZVW5GX7YURDKYLDBLOSRJPY7MYOVL5W7LVJ3QV75L53FAEQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.830564760 (291h38m26.382035221s) [noise=true , address=145.79.227.31:4798 , piece_id=DDWZBNIJSDLNGZXNUSO644NYGG5SNMV7NE4K6L56DPPT3XTAWOZQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.830886302 (291h38m26.382356763s) [noise=true , address=145.79.227.14:4839 , piece_id=FG26V3C4G3RKOTNPI4HBWY3HD6YQNGF2W5PJVKC5OQPTBDYGZYYQ, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.831939898 (291h38m26.383410359s) [noise=true , address=145.79.227.11:4730 , piece_id=NF243ZYVKX557RFPWNTY6LY3AR3VCQJ3BMFQIUOIV6YRAGEJJTAA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.832166560 (291h38m26.383637021s) [noise=true , address=77.90.31.11:4554 , piece_id=VUDSHBH7QY5IQ57UP4SRKYZ5QCHRRYWU3E4ZDAZTF65LU3ME2WOA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.833257047 (291h38m26.384727508s) [noise=true , address=145.79.227.27:4689 , piece_id=6INJ622RXLRR3JMMWVFEJKEOCQKNUBM4CCHID7V7FIRDPJ2EHUPA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:27.834108598 (291h38m26.385579059s) [noise=true , address=145.79.227.28:4726 , piece_id=VNCH5X2XDC7U7I2EICAU66EPJ74B4LZGXQDAFKE4HREPMCL7XHTA, seg_pos={0 0}, upload=256832]: Uploading piece... Nov 21 11:42:29.443724882 (291h38m27.995195343s) [upload=256832]: Next returned nil segment Nov 21 11:42:29.443741404 (291h38m27.995211875s) [upload=256832]: Waiting for error group managing segments... Nov 21 11:42:29.825233406 (291h38m28.376703857s) [noise=true , address=145.79.227.30:4762 , piece_id=X4U5XOY3WCTLSFVMPO46IYXWLS7CXBKXVYJTR5M3EYUSN4H4FMIQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.828134501 (291h38m28.379604962s) [noise=true , address=145.79.227.31:4798 , piece_id=DDWZBNIJSDLNGZXNUSO644NYGG5SNMV7NE4K6L56DPPT3XTAWOZQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.828916299 (291h38m28.38038676s) [noise=true , address=145.79.227.25:4617 , piece_id=BGWXWPS6PRMX7QY4526E6ETXZ2ADNFJ6R3N67YXYUPMZU72E2BTA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.830188623 (291h38m28.381659084s) [noise=true , address=145.79.227.20:4439 , piece_id=D4TGIE7NGGWLO3Z4PUTS45BETLZYMWKWGZUZWV27N6YSPZVUUM7A, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.831138010 (291h38m28.382608471s) [noise=true , address=145.79.227.11:4730 , piece_id=NF243ZYVKX557RFPWNTY6LY3AR3VCQJ3BMFQIUOIV6YRAGEJJTAA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.831608286 (291h38m28.383078737s) [noise=true , address=145.79.227.20:4437 , piece_id=6NLNBHESW2YOFY6NPNLIY4OJYSGM4TYO6DD6Z3A5P2CSFV4EH2RQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.839411715 (291h38m28.390882166s) [noise=true , address=145.79.227.14:4839 , piece_id=FG26V3C4G3RKOTNPI4HBWY3HD6YQNGF2W5PJVKC5OQPTBDYGZYYQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.839834861 (291h38m28.391305332s) [noise=true , address=145.79.227.22:4509 , piece_id=XFR3JCYUNAA6XNRALD2FKC3MAWUCP7SZTLO64YFQO3DTLNFBFQFA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.846124898 (291h38m28.397595359s) [noise=true , address=145.79.227.27:4689 , piece_id=6INJ622RXLRR3JMMWVFEJKEOCQKNUBM4CCHID7V7FIRDPJ2EHUPA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:29.854921609 (291h38m28.40639206s) [noise=true , address=145.79.227.28:4726 , piece_id=VNCH5X2XDC7U7I2EICAU66EPJ74B4LZGXQDAFKE4HREPMCL7XHTA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.404759674 (291h38m28.956230146s) [noise=true , address=45.91.116.8:4048 , piece_id=2N4UQ5UNTKSQQMFZEG2UFSNE5AISQ7I4O5JJYX7TPZN32BUG4TDA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.531857242 (291h38m29.083327693s) [noise=true , address=45.91.116.14:4190 , piece_id=FRCQYZVW5GX7YURDKYLDBLOSRJPY7MYOVL5W7LVJ3QV75L53FAEQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.758748116 (291h38m29.310218577s) [noise=true , address=77.90.31.11:4554 , piece_id=VUDSHBH7QY5IQ57UP4SRKYZ5QCHRRYWU3E4ZDAZTF65LU3ME2WOA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: <nil> Nov 21 11:42:30.758774556 (291h38m29.310245017s) [seg_pos={0 0}, upload=256832]: Segment reached optimal threshold of 13 pieces. Nov 21 11:42:30.759205336 (291h38m29.310675797s) [noise=true , address=77.90.31.17:4770 , piece_id=RWJFMY524OVK57SNMQ7YFMPQRID6QS2SHUX57KE6PBIRDTVRI6XQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Nov 21 11:42:30.759245923 (291h38m29.310716374s) [noise=true , address=77.90.31.17:4770 , piece_id=RWJFMY524OVK57SNMQ7YFMPQRID6QS2SHUX57KE6PBIRDTVRI6XQ, seg_pos={0 0}, upload=256832]: Long tail cancellation due to optimal threshold Nov 21 11:42:30.759290920 (291h38m29.310761381s) [noise=true , address=77.90.31.15:4697 , piece_id=ZXQUVQZSN27ML5WQ2GOO3BB5G2U75CLEVNSSBG6MZRNH4D6IRVXA, seg_pos={0 0}, upload=256832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Nov 21 11:42:30.759301820 (291h38m29.310772281s) [noise=true , address=77.90.31.15:4697 , piece_id=ZXQUVQZSN27ML5WQ2GOO3BB5G2U75CLEVNSSBG6MZRNH4D6IRVXA, seg_pos={0 0}, upload=256832]: Long tail cancellation due to optimal threshold Nov 21 11:42:30.759587844 (291h38m29.311058306s) [noise=true , address=45.91.116.4:4940 , piece_id=DTQHJJGXOXE6B2VHK5JEMAVKOSWNDRTG3CDGSBKMQI3AOV3UWPMQ, seg_pos={0 0}, upload=256832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: write tcp 188.166.225.170:56388->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:56388->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56388->45.91.116.4:4940: use of closed network connection Nov 21 11:42:30.759598234 (291h38m29.311068695s) [noise=true , address=45.91.116.4:4940 , piece_id=DTQHJJGXOXE6B2VHK5JEMAVKOSWNDRTG3CDGSBKMQI3AOV3UWPMQ, seg_pos={0 0}, upload=256832]: Long tail cancellation due to optimal threshold Nov 21 11:42:30.759624644 (291h38m29.311095106s) [seg_pos={0 0}, upload=256832]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:42:30.759725727 (291h38m29.311196188s) [upload=256832]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:42:30.759734524 (291h38m29.311204985s) [upload=256832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:42:31.046104166 (291h38m29.597574617s) [upload=256832]: Done uploading segments... Nov 21 11:42:31.046167967 (291h38m29.597638418s) [upload=256832]: Upload finished. err: <nil> Duration: 30.003455311s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 259.930998ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:42:47 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 259.960423ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 257.491764ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:42:41 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 257.515649ms edge linksharing bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [...XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX........] duration: 3.369842511s trace: https://jaeger.dev.storj.io/trace/6af15a3a67b655d5 FAIL (2025-11-21 11:42:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.369868s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 257.042136ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:42:29 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 257.070812ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 242.350201ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:42:27 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 242.37647ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 258.090173ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:42:24 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 258.121573ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.X........................] duration: 5.927953887s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:42:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 5.927973805s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 255.927495ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:42:02 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 255.989943ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...........X...X.X......X.XX...X.X.X....X............X....X.] duration: 6.429929876s trace: https://jaeger.dev.storj.io/trace/69e1ebd1c45373c FAIL (2025-11-21 11:41:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 3BI22NJKKK46PRFZ7MQ5VRRUMWTMCX2BU5DUVUF2XMGNWUZKBOLQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: MAXQEDRM3PAP3VTYXCP4UPZKMJ7EGYW7KE3ZX2EKCZBDJGNQ24WQ) context deadline exceeded ======LOGS====== Nov 21 11:40:44.490365952 (291h36m43.041836423s) [upload=256808]: Starting upload Nov 21 11:40:44.490456043 (291h36m43.041926504s) [upload=256808]: Done starting upload Nov 21 11:40:44.490848269 (291h36m43.04231874s) [upload=256808]: Uploading segments... Nov 21 11:40:44.490910358 (291h36m43.042380829s) [upload=256808]: Got next segment. Inline: false Nov 21 11:40:44.490949793 (291h36m43.042420274s) [upload=256808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:40:44.490956717 (291h36m43.042427198s) [upload=256808]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:40:44.735648634 (291h36m43.287119095s) [seg_pos={0 0}, upload=256808]: Begin upload segment... Nov 21 11:40:44.735702587 (291h36m43.287173048s) [seg_pos={0 0}, upload=256808]: Done begin upload segment. Nov 21 11:40:44.735744437 (291h36m43.287214908s) [noise=true , address=77.90.31.16:4734 , piece_id=VUJDCDUQ3K5FIMDDN2Q3GI5VFXBUZJTPB5RDJYXIUY2YDYVZFWNA, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.735847072 (291h36m43.287317533s) [noise=true , address=145.79.227.8:4331 , piece_id=A26GAR4UEBCZ2DCSS3XUSV3AXDQDJ4W5L4UROJXK7GABH44BGBMA, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.736447876 (291h36m43.287918327s) [noise=true , address=145.79.227.15:4875 , piece_id=SDA2NA6L7A37XBPXCOJVZAOL4JHT3E4774XZM6VVNDZKZO3FCDHQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.736775801 (291h36m43.288246262s) [noise=true , address=77.90.31.10:4519 , piece_id=3BI22NJKKK46PRFZ7MQ5VRRUMWTMCX2BU5DUVUF2XMGNWUZKBOLQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.737089789 (291h36m43.28856025s) [noise=true , address=145.79.227.11:4730 , piece_id=FD5CGPBY3EPFNQJLF2IAZAUSZ4XSHDZVSRAUDNGOLUDFZRSTML4A, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.737427411 (291h36m43.288897892s) [noise=true , address=45.91.116.7:4009 , piece_id=44SOTGVCG4ZAUNMNNGFBPJGI464X6E4XXEOOZ3W2KJYPFYIM6UHQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.737922874 (291h36m43.289393325s) [noise=true , address=77.90.31.19:4841 , piece_id=H4DS5OXB4H636NMNKQ3O3BPBPAIMWKD3JLF4JZBYD4HCLZYZZEQQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.738266609 (291h36m43.28973706s) [noise=true , address=145.79.227.25:4618 , piece_id=QOKGVWIN2JF3E2YBGCEJR5ORVRMWUXA2K4Q6RMA4MWCJ7J5Z65PQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.738800386 (291h36m43.290270837s) [noise=true , address=45.91.116.10:4082 , piece_id=MRQS5HQWNHDD5QFM47OTFX5TDFZSU5JPHO7RT2Z2AFGEOALZZ44Q, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.739403194 (291h36m43.290873655s) [noise=true , address=145.79.227.28:4726 , piece_id=ULYH4ZQUNP7RBA7WEWFUDRP3A5BBOLNAZFKP7BS6JBDN44PSTX3A, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.739811351 (291h36m43.291281812s) [noise=true , address=145.79.227.30:4763 , piece_id=QDYPZHXOVU5BEGZJFOLZV7FKSGI37HNUVOCOHQA66DFGVSSDAIAQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.740155647 (291h36m43.291626098s) [noise=true , address=45.91.116.14:4190 , piece_id=MAXQEDRM3PAP3VTYXCP4UPZKMJ7EGYW7KE3ZX2EKCZBDJGNQ24WQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.740623147 (291h36m43.292093598s) [noise=true , address=145.79.227.11:4731 , piece_id=4EFQX5WQGOB32XDSDDVLJE6276KS6I43OAFK22H42V7ZNCIX7CNA, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.741905819 (291h36m43.29337628s) [noise=true , address=145.79.227.31:4799 , piece_id=CWVNBV2KKS4CSUGALZXS3T4NSOD4WSWBFEDRGSSK3WUWSZT3EPRQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.747098617 (291h36m43.298569078s) [noise=true , address=145.79.227.24:4581 , piece_id=OTKFUS6CNHE2HUSOPAFUB7E7MQ2ZKJE5YC3U27TFGKAVWJRM6YJQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.748709514 (291h36m43.300179975s) [noise=true , address=145.79.227.14:4839 , piece_id=NX6AERKM6L2RX3JN5LZQ4EBBDTL6BE2ZLL4B25JZJU2DYN2LNDOQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:46.443828492 (291h36m44.995298953s) [upload=256808]: Next returned nil segment Nov 21 11:40:46.443844232 (291h36m44.995314694s) [upload=256808]: Waiting for error group managing segments... Nov 21 11:40:46.724509292 (291h36m45.275979753s) [noise=true , address=145.79.227.11:4731 , piece_id=4EFQX5WQGOB32XDSDDVLJE6276KS6I43OAFK22H42V7ZNCIX7CNA, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.731925745 (291h36m45.283396206s) [noise=true , address=145.79.227.11:4730 , piece_id=FD5CGPBY3EPFNQJLF2IAZAUSZ4XSHDZVSRAUDNGOLUDFZRSTML4A, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.732400628 (291h36m45.283871089s) [noise=true , address=145.79.227.8:4331 , piece_id=A26GAR4UEBCZ2DCSS3XUSV3AXDQDJ4W5L4UROJXK7GABH44BGBMA, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.734032606 (291h36m45.285503067s) [noise=true , address=145.79.227.24:4581 , piece_id=OTKFUS6CNHE2HUSOPAFUB7E7MQ2ZKJE5YC3U27TFGKAVWJRM6YJQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.741638760 (291h36m45.293109221s) [noise=true , address=145.79.227.25:4618 , piece_id=QOKGVWIN2JF3E2YBGCEJR5ORVRMWUXA2K4Q6RMA4MWCJ7J5Z65PQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.746594867 (291h36m45.298065328s) [noise=true , address=145.79.227.28:4726 , piece_id=ULYH4ZQUNP7RBA7WEWFUDRP3A5BBOLNAZFKP7BS6JBDN44PSTX3A, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.752194069 (291h36m45.30366453s) [noise=true , address=145.79.227.15:4875 , piece_id=SDA2NA6L7A37XBPXCOJVZAOL4JHT3E4774XZM6VVNDZKZO3FCDHQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.753504845 (291h36m45.304975306s) [noise=true , address=145.79.227.14:4839 , piece_id=NX6AERKM6L2RX3JN5LZQ4EBBDTL6BE2ZLL4B25JZJU2DYN2LNDOQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.755097948 (291h36m45.306568399s) [noise=true , address=145.79.227.30:4763 , piece_id=QDYPZHXOVU5BEGZJFOLZV7FKSGI37HNUVOCOHQA66DFGVSSDAIAQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.757192828 (291h36m45.308663279s) [noise=true , address=145.79.227.31:4799 , piece_id=CWVNBV2KKS4CSUGALZXS3T4NSOD4WSWBFEDRGSSK3WUWSZT3EPRQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.214949243 (291h36m45.766419704s) [noise=true , address=45.91.116.14:4190 , piece_id=MAXQEDRM3PAP3VTYXCP4UPZKMJ7EGYW7KE3ZX2EKCZBDJGNQ24WQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.266695457 (291h36m45.818165918s) [noise=true , address=77.90.31.10:4519 , piece_id=3BI22NJKKK46PRFZ7MQ5VRRUMWTMCX2BU5DUVUF2XMGNWUZKBOLQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.321163685 (291h36m45.872634156s) [noise=true , address=77.90.31.16:4734 , piece_id=VUJDCDUQ3K5FIMDDN2Q3GI5VFXBUZJTPB5RDJYXIUY2YDYVZFWNA, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.321192209 (291h36m45.87266267s) [seg_pos={0 0}, upload=256808]: Segment reached optimal threshold of 13 pieces. Nov 21 11:40:47.321495266 (291h36m45.872965728s) [noise=true , address=45.91.116.10:4082 , piece_id=MRQS5HQWNHDD5QFM47OTFX5TDFZSU5JPHO7RT2Z2AFGEOALZZ44Q, seg_pos={0 0}, upload=256808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Nov 21 11:40:47.321505776 (291h36m45.872976227s) [noise=true , address=45.91.116.10:4082 , piece_id=MRQS5HQWNHDD5QFM47OTFX5TDFZSU5JPHO7RT2Z2AFGEOALZZ44Q, seg_pos={0 0}, upload=256808]: Long tail cancellation due to optimal threshold Nov 21 11:40:47.321898474 (291h36m45.873368945s) [noise=true , address=77.90.31.19:4841 , piece_id=H4DS5OXB4H636NMNKQ3O3BPBPAIMWKD3JLF4JZBYD4HCLZYZZEQQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:44642->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:44642->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44642->77.90.31.19:4841: use of closed network connection Nov 21 11:40:47.321910357 (291h36m45.873380818s) [noise=true , address=77.90.31.19:4841 , piece_id=H4DS5OXB4H636NMNKQ3O3BPBPAIMWKD3JLF4JZBYD4HCLZYZZEQQ, seg_pos={0 0}, upload=256808]: Long tail cancellation due to optimal threshold Nov 21 11:40:47.321934122 (291h36m45.873404653s) [noise=true , address=45.91.116.7:4009 , piece_id=44SOTGVCG4ZAUNMNNGFBPJGI464X6E4XXEOOZ3W2KJYPFYIM6UHQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Nov 21 11:40:47.321943901 (291h36m45.873414362s) [noise=true , address=45.91.116.7:4009 , piece_id=44SOTGVCG4ZAUNMNNGFBPJGI464X6E4XXEOOZ3W2KJYPFYIM6UHQ, seg_pos={0 0}, upload=256808]: Long tail cancellation due to optimal threshold Nov 21 11:40:47.322083577 (291h36m45.873554048s) [seg_pos={0 0}, upload=256808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:40:47.322197053 (291h36m45.873667514s) [upload=256808]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:40:47.322204707 (291h36m45.873675168s) [upload=256808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:40:47.613510899 (291h36m46.16498137s) [upload=256808]: Done uploading segments... Nov 21 11:40:47.613540816 (291h36m46.165011277s) [upload=256808]: Upload finished. err: <nil> Duration: 6.429961175s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [....XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.......] duration: 3.46835235s trace: https://jaeger.dev.storj.io/trace/6af15a3a67b655d5 FAIL (2025-11-21 11:41:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.468379351s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 256.133005ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:41:27 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 256.161139ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 256.287569ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:41:20 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 256.31355ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 237.802651ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:41:20 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 237.83356ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 235.811951ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:41:18 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 235.836679ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [............X...X.X......X.XX...X.X.X....X............X....X] duration: 30.002662088s trace: https://jaeger.dev.storj.io/trace/69e1ebd1c45373c FAIL (2025-11-21 11:41:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 3BI22NJKKK46PRFZ7MQ5VRRUMWTMCX2BU5DUVUF2XMGNWUZKBOLQ) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: MAXQEDRM3PAP3VTYXCP4UPZKMJ7EGYW7KE3ZX2EKCZBDJGNQ24WQ) context deadline exceeded ======LOGS====== Nov 21 11:40:44.490365952 (291h36m43.041836423s) [upload=256808]: Starting upload Nov 21 11:40:44.490456043 (291h36m43.041926504s) [upload=256808]: Done starting upload Nov 21 11:40:44.490848269 (291h36m43.04231874s) [upload=256808]: Uploading segments... Nov 21 11:40:44.490910358 (291h36m43.042380829s) [upload=256808]: Got next segment. Inline: false Nov 21 11:40:44.490949793 (291h36m43.042420274s) [upload=256808]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 21 11:40:44.490956717 (291h36m43.042427198s) [upload=256808]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 21 11:40:44.735648634 (291h36m43.287119095s) [seg_pos={0 0}, upload=256808]: Begin upload segment... Nov 21 11:40:44.735702587 (291h36m43.287173048s) [seg_pos={0 0}, upload=256808]: Done begin upload segment. Nov 21 11:40:44.735744437 (291h36m43.287214908s) [noise=true , address=77.90.31.16:4734 , piece_id=VUJDCDUQ3K5FIMDDN2Q3GI5VFXBUZJTPB5RDJYXIUY2YDYVZFWNA, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.735847072 (291h36m43.287317533s) [noise=true , address=145.79.227.8:4331 , piece_id=A26GAR4UEBCZ2DCSS3XUSV3AXDQDJ4W5L4UROJXK7GABH44BGBMA, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.736447876 (291h36m43.287918327s) [noise=true , address=145.79.227.15:4875 , piece_id=SDA2NA6L7A37XBPXCOJVZAOL4JHT3E4774XZM6VVNDZKZO3FCDHQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.736775801 (291h36m43.288246262s) [noise=true , address=77.90.31.10:4519 , piece_id=3BI22NJKKK46PRFZ7MQ5VRRUMWTMCX2BU5DUVUF2XMGNWUZKBOLQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.737089789 (291h36m43.28856025s) [noise=true , address=145.79.227.11:4730 , piece_id=FD5CGPBY3EPFNQJLF2IAZAUSZ4XSHDZVSRAUDNGOLUDFZRSTML4A, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.737427411 (291h36m43.288897892s) [noise=true , address=45.91.116.7:4009 , piece_id=44SOTGVCG4ZAUNMNNGFBPJGI464X6E4XXEOOZ3W2KJYPFYIM6UHQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.737922874 (291h36m43.289393325s) [noise=true , address=77.90.31.19:4841 , piece_id=H4DS5OXB4H636NMNKQ3O3BPBPAIMWKD3JLF4JZBYD4HCLZYZZEQQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.738266609 (291h36m43.28973706s) [noise=true , address=145.79.227.25:4618 , piece_id=QOKGVWIN2JF3E2YBGCEJR5ORVRMWUXA2K4Q6RMA4MWCJ7J5Z65PQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.738800386 (291h36m43.290270837s) [noise=true , address=45.91.116.10:4082 , piece_id=MRQS5HQWNHDD5QFM47OTFX5TDFZSU5JPHO7RT2Z2AFGEOALZZ44Q, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.739403194 (291h36m43.290873655s) [noise=true , address=145.79.227.28:4726 , piece_id=ULYH4ZQUNP7RBA7WEWFUDRP3A5BBOLNAZFKP7BS6JBDN44PSTX3A, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.739811351 (291h36m43.291281812s) [noise=true , address=145.79.227.30:4763 , piece_id=QDYPZHXOVU5BEGZJFOLZV7FKSGI37HNUVOCOHQA66DFGVSSDAIAQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.740155647 (291h36m43.291626098s) [noise=true , address=45.91.116.14:4190 , piece_id=MAXQEDRM3PAP3VTYXCP4UPZKMJ7EGYW7KE3ZX2EKCZBDJGNQ24WQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.740623147 (291h36m43.292093598s) [noise=true , address=145.79.227.11:4731 , piece_id=4EFQX5WQGOB32XDSDDVLJE6276KS6I43OAFK22H42V7ZNCIX7CNA, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.741905819 (291h36m43.29337628s) [noise=true , address=145.79.227.31:4799 , piece_id=CWVNBV2KKS4CSUGALZXS3T4NSOD4WSWBFEDRGSSK3WUWSZT3EPRQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.747098617 (291h36m43.298569078s) [noise=true , address=145.79.227.24:4581 , piece_id=OTKFUS6CNHE2HUSOPAFUB7E7MQ2ZKJE5YC3U27TFGKAVWJRM6YJQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:44.748709514 (291h36m43.300179975s) [noise=true , address=145.79.227.14:4839 , piece_id=NX6AERKM6L2RX3JN5LZQ4EBBDTL6BE2ZLL4B25JZJU2DYN2LNDOQ, seg_pos={0 0}, upload=256808]: Uploading piece... Nov 21 11:40:46.443828492 (291h36m44.995298953s) [upload=256808]: Next returned nil segment Nov 21 11:40:46.443844232 (291h36m44.995314694s) [upload=256808]: Waiting for error group managing segments... Nov 21 11:40:46.724509292 (291h36m45.275979753s) [noise=true , address=145.79.227.11:4731 , piece_id=4EFQX5WQGOB32XDSDDVLJE6276KS6I43OAFK22H42V7ZNCIX7CNA, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.731925745 (291h36m45.283396206s) [noise=true , address=145.79.227.11:4730 , piece_id=FD5CGPBY3EPFNQJLF2IAZAUSZ4XSHDZVSRAUDNGOLUDFZRSTML4A, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.732400628 (291h36m45.283871089s) [noise=true , address=145.79.227.8:4331 , piece_id=A26GAR4UEBCZ2DCSS3XUSV3AXDQDJ4W5L4UROJXK7GABH44BGBMA, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.734032606 (291h36m45.285503067s) [noise=true , address=145.79.227.24:4581 , piece_id=OTKFUS6CNHE2HUSOPAFUB7E7MQ2ZKJE5YC3U27TFGKAVWJRM6YJQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.741638760 (291h36m45.293109221s) [noise=true , address=145.79.227.25:4618 , piece_id=QOKGVWIN2JF3E2YBGCEJR5ORVRMWUXA2K4Q6RMA4MWCJ7J5Z65PQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.746594867 (291h36m45.298065328s) [noise=true , address=145.79.227.28:4726 , piece_id=ULYH4ZQUNP7RBA7WEWFUDRP3A5BBOLNAZFKP7BS6JBDN44PSTX3A, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.752194069 (291h36m45.30366453s) [noise=true , address=145.79.227.15:4875 , piece_id=SDA2NA6L7A37XBPXCOJVZAOL4JHT3E4774XZM6VVNDZKZO3FCDHQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.753504845 (291h36m45.304975306s) [noise=true , address=145.79.227.14:4839 , piece_id=NX6AERKM6L2RX3JN5LZQ4EBBDTL6BE2ZLL4B25JZJU2DYN2LNDOQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.755097948 (291h36m45.306568399s) [noise=true , address=145.79.227.30:4763 , piece_id=QDYPZHXOVU5BEGZJFOLZV7FKSGI37HNUVOCOHQA66DFGVSSDAIAQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:46.757192828 (291h36m45.308663279s) [noise=true , address=145.79.227.31:4799 , piece_id=CWVNBV2KKS4CSUGALZXS3T4NSOD4WSWBFEDRGSSK3WUWSZT3EPRQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.214949243 (291h36m45.766419704s) [noise=true , address=45.91.116.14:4190 , piece_id=MAXQEDRM3PAP3VTYXCP4UPZKMJ7EGYW7KE3ZX2EKCZBDJGNQ24WQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.266695457 (291h36m45.818165918s) [noise=true , address=77.90.31.10:4519 , piece_id=3BI22NJKKK46PRFZ7MQ5VRRUMWTMCX2BU5DUVUF2XMGNWUZKBOLQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.321163685 (291h36m45.872634156s) [noise=true , address=77.90.31.16:4734 , piece_id=VUJDCDUQ3K5FIMDDN2Q3GI5VFXBUZJTPB5RDJYXIUY2YDYVZFWNA, seg_pos={0 0}, upload=256808]: Done uploading piece. err: <nil> Nov 21 11:40:47.321192209 (291h36m45.87266267s) [seg_pos={0 0}, upload=256808]: Segment reached optimal threshold of 13 pieces. Nov 21 11:40:47.321495266 (291h36m45.872965728s) [noise=true , address=45.91.116.10:4082 , piece_id=MRQS5HQWNHDD5QFM47OTFX5TDFZSU5JPHO7RT2Z2AFGEOALZZ44Q, seg_pos={0 0}, upload=256808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Nov 21 11:40:47.321505776 (291h36m45.872976227s) [noise=true , address=45.91.116.10:4082 , piece_id=MRQS5HQWNHDD5QFM47OTFX5TDFZSU5JPHO7RT2Z2AFGEOALZZ44Q, seg_pos={0 0}, upload=256808]: Long tail cancellation due to optimal threshold Nov 21 11:40:47.321898474 (291h36m45.873368945s) [noise=true , address=77.90.31.19:4841 , piece_id=H4DS5OXB4H636NMNKQ3O3BPBPAIMWKD3JLF4JZBYD4HCLZYZZEQQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:44642->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:44642->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44642->77.90.31.19:4841: use of closed network connection Nov 21 11:40:47.321910357 (291h36m45.873380818s) [noise=true , address=77.90.31.19:4841 , piece_id=H4DS5OXB4H636NMNKQ3O3BPBPAIMWKD3JLF4JZBYD4HCLZYZZEQQ, seg_pos={0 0}, upload=256808]: Long tail cancellation due to optimal threshold Nov 21 11:40:47.321934122 (291h36m45.873404653s) [noise=true , address=45.91.116.7:4009 , piece_id=44SOTGVCG4ZAUNMNNGFBPJGI464X6E4XXEOOZ3W2KJYPFYIM6UHQ, seg_pos={0 0}, upload=256808]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Nov 21 11:40:47.321943901 (291h36m45.873414362s) [noise=true , address=45.91.116.7:4009 , piece_id=44SOTGVCG4ZAUNMNNGFBPJGI464X6E4XXEOOZ3W2KJYPFYIM6UHQ, seg_pos={0 0}, upload=256808]: Long tail cancellation due to optimal threshold Nov 21 11:40:47.322083577 (291h36m45.873554048s) [seg_pos={0 0}, upload=256808]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 21 11:40:47.322197053 (291h36m45.873667514s) [upload=256808]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:40:47.322204707 (291h36m45.873675168s) [upload=256808]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:40:47.613510899 (291h36m46.16498137s) [upload=256808]: Done uploading segments... Nov 21 11:40:47.613540816 (291h36m46.165011277s) [upload=256808]: Upload finished. err: <nil> Duration: 30.002685512s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.X.......................] duration: 6.375842093s trace: https://jaeger.dev.storj.io/trace/a7f69ee106efb7f FAIL (2025-11-21 11:41:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== upload=256586]: Uploading piece... Nov 21 11:25:27.210346024 (291h21m25.761816485s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.211425289 (291h21m25.76289575s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213103795 (291h21m25.764574246s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.213408395 (291h21m25.764878856s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.214501747 (291h21m25.765972208s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215013752 (291h21m25.766484213s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.215087572 (291h21m25.766558044s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216432844 (291h21m25.767903305s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.216662281 (291h21m25.768132742s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:27.218008314 (291h21m25.769478775s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Uploading piece... Nov 21 11:25:28.035150976 (291h21m26.586621437s) [upload=256586]: Next returned nil segment Nov 21 11:25:28.035169622 (291h21m26.586640083s) [upload=256586]: Waiting for error group managing segments... Nov 21 11:25:28.416646836 (291h21m26.968117317s) [noise=true , address=185.167.204.206:4064 , piece_id=SRFIWFQLRAGCNIKYLHMGT4QRUL3WH2QPLLIKAJ7DLFJEQUW5DPPA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.417338854 (291h21m26.968809325s) [noise=true , address=185.167.204.205:4033 , piece_id=FEWQVHPIKN3Q6O5AZ6AGJBLUPZ34PBZFB5X3XNSKGPOGI4AIO7JA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.421089406 (291h21m26.972559867s) [noise=true , address=185.167.204.206:4072 , piece_id=GYCO53OWGOMOHK4FHC5GENN4LTQGSZKZKOI5V534RLV3LHLPKC6Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.426252887 (291h21m26.977723348s) [noise=true , address=185.167.204.206:4046 , piece_id=BZ57ZA35ITEOA6CAZJGNODK5MUTXEZWDV7E2OGBZOSMS4OQZYAEA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.431720689 (291h21m26.98319115s) [noise=true , address=185.167.204.205:4027 , piece_id=IVPONKBF2PIHZQAW3L4YA4HOMQV5BQML6SKKCJLIEGT5SZSQ2AOA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.438243951 (291h21m26.989714412s) [noise=true , address=185.167.204.205:4022 , piece_id=BES3YK6PYQV2AKOWSAGHF7DGGWGWSWPIJU2ICAQ5WBHKZ5VT5L6A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.443364331 (291h21m26.994834792s) [noise=true , address=185.167.204.206:4056 , piece_id=EAMZIK6AVJL7IMFF5VOLW3SMCGYKSKB2VXJTNUNLJCVIT7EKXNMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.445594728 (291h21m26.997065179s) [noise=true , address=185.167.204.205:4016 , piece_id=EHQS3546UXRXTL2MGFCSRTPDDPAYXEYLMISPP5AWY337MHX7HFLQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449325822 (291h21m27.000796283s) [noise=true , address=185.167.204.205:4031 , piece_id=ZEI2YSIQC655R7WS43WADXGFT5RKT5LFSN6IZEQ6SHTDVOV7BZBQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.449761211 (291h21m27.001231672s) [noise=true , address=185.167.204.205:4018 , piece_id=7OSLPWJLKFTIJPZEAFMZOAH6JAAMX6HKROI25FODFYBPNJWSMLWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.450419845 (291h21m27.001890306s) [noise=true , address=185.167.204.206:4062 , piece_id=UF3S6NAVGS2KQJPGA5WXP7YG4RQO44P445SNPCWXKXMONZAYUTNQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:28.453201252 (291h21m27.004671713s) [noise=true , address=185.167.204.205:4007 , piece_id=MEBGOPZ6MFMQY4UZARS2LKJKEMNN5YGSLKPJGTWSKS45CUNXSRGQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.661209626 (291h21m28.212680087s) [noise=true , address=89.149.201.147:4016 , piece_id=TGRWGGGBKDANC7YKROWPRMIYNAVFEVE3RXOSUOINFI7KTRS2IYUQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.693169182 (291h21m28.244639643s) [noise=true , address=89.149.201.147:4002 , piece_id=UIYJSCMY3LCLRHZUJB6JFJHHXKMYTVKERTZW7F7X44MWJETPY6XA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.721204497 (291h21m28.272674958s) [noise=true , address=89.149.201.147:4008 , piece_id=HCX7I4ICPEHDWTYDFIQMKZ5EE5CGPQLYAIHF7WZMTUUU6CNZPHTA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.764584711 (291h21m28.316055172s) [noise=true , address=89.149.201.147:4001 , piece_id=QUS6NB3NQBLJFB6YCK3EBDPKCTWEWGKAT3NOR24TVI65SQKHWDOQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.768284075 (291h21m28.319754536s) [noise=true , address=89.149.201.147:4011 , piece_id=ZATNMD5HOK35AWGJJ5V66VDJ4YGUVZQJ2OBERQL32ZV5GW2HBSMQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:29.789681961 (291h21m28.341152422s) [noise=true , address=89.149.201.147:4003 , piece_id=RUAYGN2RGF5JJTWFEGIHWH3BHEUDSS5DN2QJ6YLSHPF5BGUXD3RA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.298598682 (291h21m35.850069143s) [noise=true , address=81.171.14.70:4043 , piece_id=T3NF6JR3VJIGDMPNUBLMJ3AOTQP5IHOGJYORMCSS7ZMOENQETZ7A, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:37.497478356 (291h21m36.048948818s) [noise=true , address=81.171.14.70:4032 , piece_id=CFH2RTR3BLBUB5ZIZCFC6UHT3FHT3B2XSCHQMKXNGDJJGPHEMCTQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:40.406166933 (291h21m38.957637394s) [noise=true , address=81.171.14.70:4045 , piece_id=BRWQ7G3E547PJE2RLGSESIB3HMP5C4M5NSKAJNGWSFXUBEO6SYEQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:25:42.511041157 (291h21m41.062511608s) [noise=true , address=81.171.14.70:4029 , piece_id=DKJM5E7EY7LOSBSEBQOZRC6CI5ZPPE47HFOFQJROPCUMUV567CAA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:12.012682521 (291h22m10.564152982s) [noise=true , address=81.171.14.70:4042 , piece_id=XNREZEUZ5L5KZ3L7ZOL6IRK7IHU6DJN4RLEQZRFMDKDH2JWC24CQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:15.596046283 (291h22m14.147516744s) [noise=true , address=81.171.14.70:4033 , piece_id=JRKPD4EP6FZM5HISOIDJYZY7FAWWQCADAS47FGW2JM5A7RFJR2BQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:18.453059659 (291h22m17.00453012s) [noise=true , address=81.171.14.70:4046 , piece_id=OSONRSF6KJS7F2QJVCC3FPZK4AV7QPG5YSM47CW5LIW7EME44AWA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442583344 (291h22m21.994053805s) [noise=true , address=81.171.14.70:4034 , piece_id=ITRLKYVNFSYBLABNGNFO3HRJNMX3MWB345FIMWSIPBNAVZPA47UA, seg_pos={0 0}, upload=256586]: Done uploading piece. err: <nil> Nov 21 11:26:23.442621967 (291h22m21.994092428s) [seg_pos={0 0}, upload=256586]: Segment reached optimal threshold of 26 pieces. Nov 21 11:26:23.442836976 (291h22m21.994307437s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.442845934 (291h22m21.994316395s) [noise=true , address=81.171.14.70:4044 , piece_id=QYUBNGS2FAFHIWMW7GAGYJ3UVIWVP62LIDCOK6QUXEIQNIXRJDAQ, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443227491 (291h22m21.994697942s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4): protocol: expected piece hash; context canceled; EOF Nov 21 11:26:23.443237770 (291h22m21.994708231s) [noise=true , address=81.171.14.70:4027 , piece_id=4FDFCCWF3EIROQAFLOHGS743523OYGSXWQSPQEVD3AMQ6WKDQI7Q, seg_pos={0 0}, upload=256586]: Long tail cancellation due to optimal threshold Nov 21 11:26:23.443274720 (291h22m21.994745181s) [seg_pos={0 0}, upload=256586]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 11:26:23.443378929 (291h22m21.99484939s) [upload=256586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 11:26:23.443387044 (291h22m21.994857495s) [upload=256586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 11:26:23.685139626 (291h22m22.236610077s) [upload=256586]: Done uploading segments... Nov 21 11:26:23.685160676 (291h22m22.236631127s) [upload=256586]: Upload finished. err: <nil> Duration: 6.375873342s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 255.655448ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:40:55 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 255.685284ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 260.590775ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:40:41 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 260.617195ms edge linksharing bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)