Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.120594463s FAIL (2025-10-16 17:54:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.120656842s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.078556895s FAIL (2025-10-16 17:53:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.078584158s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............................X..............X................] duration: 5.002951451s FAIL (2025-10-16 17:53:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.00297662s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.128046941s FAIL (2025-10-16 17:53:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.1280726s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.384032735s FAIL (2025-10-16 17:53:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.384054999s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.211413609s FAIL (2025-10-16 17:53:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.211448477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.218347552s FAIL (2025-10-16 17:53:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.218374964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.569276181s FAIL (2025-10-16 17:53:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.569310448s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............................X..............X...............] duration: 4.975082142s FAIL (2025-10-16 17:53:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.975117158s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.320150594s FAIL (2025-10-16 17:52:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.320256296s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.363511761s FAIL (2025-10-16 17:52:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.363543552s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............................X..............X..............] duration: 30.513048544s FAIL (2025-10-16 17:52:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 30.513105121s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.149997874s FAIL (2025-10-16 17:52:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.150119574s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.129135595s FAIL (2025-10-16 17:51:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.129172977s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.39837072s FAIL (2025-10-16 17:51:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.398404864s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.206880296s FAIL (2025-10-16 17:51:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.206988783s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.552042306s FAIL (2025-10-16 17:51:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.552087272s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..............X.............] duration: 5.581003424s FAIL (2025-10-16 17:51:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.581030195s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.446003892s FAIL (2025-10-16 17:51:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.446027666s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.387421388s FAIL (2025-10-16 17:51:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.387471844s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [................................X..............X............] duration: 39.091819248s FAIL (2025-10-16 17:51:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 39.091856178s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.275953696s FAIL (2025-10-16 17:50:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.275985607s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.569792708s FAIL (2025-10-16 17:50:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.569827734s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.148087176s FAIL (2025-10-16 17:50:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.148238222s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.................................X..............X...........] duration: 5.399368019s FAIL (2025-10-16 17:50:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.399396392s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.653554514s FAIL (2025-10-16 17:50:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.653593257s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.176498486s FAIL (2025-10-16 17:49:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.176530145s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.303581082s FAIL (2025-10-16 17:49:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.303614576s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.307948188s FAIL (2025-10-16 17:49:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.307977694s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.242096966s FAIL (2025-10-16 17:49:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.242127624s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..................................X..............X..........] duration: 5.446546252s FAIL (2025-10-16 17:49:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.446579907s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.205929967s FAIL (2025-10-16 17:48:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.205966077s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.152224324s FAIL (2025-10-16 17:48:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.152261263s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.600051986s FAIL (2025-10-16 17:48:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.600088987s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.434818373s FAIL (2025-10-16 17:48:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.434846076s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...................................X..............X.........] duration: 19.809499332s FAIL (2025-10-16 17:48:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 19.80953534s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.138953649s FAIL (2025-10-16 17:47:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.13901195s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.344070949s FAIL (2025-10-16 17:47:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.344131213s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.23586273s FAIL (2025-10-16 17:47:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.235897868s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.285171875s FAIL (2025-10-16 17:47:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.285206401s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.306782695s FAIL (2025-10-16 17:47:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.306814456s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....................................X..............X........] duration: 5.040185202s FAIL (2025-10-16 17:47:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.040228223s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.218642691s FAIL (2025-10-16 17:46:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.21872666s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.016738429s FAIL (2025-10-16 17:46:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.016778365s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.42152232s FAIL (2025-10-16 17:46:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.42154873s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X..............X.......] duration: 5.620625257s FAIL (2025-10-16 17:46:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.620660603s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.478588264s FAIL (2025-10-16 17:46:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.478627858s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.003857158s FAIL (2025-10-16 17:46:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.003893087s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.079189116s FAIL (2025-10-16 17:46:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.079225425s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.38816351s FAIL (2025-10-16 17:46:00 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.388198025s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)