Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.........X...................................X..............] duration: 4.296103091s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:22:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.296135152s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.627992067s trace: https://jaeger.dev.storj.io/trace/690c0830f44a9942 FAIL (2025-12-08 20:22:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 8 20:13:16.312406346 (76h44m16.183489391s) [upload=67101]: Starting upload Dec 8 20:13:16.312474916 (76h44m16.183557952s) [upload=67101]: Done starting upload Dec 8 20:13:16.312661411 (76h44m16.183744446s) [upload=67101]: Uploading segments... Dec 8 20:13:16.313579429 (76h44m16.184662474s) [upload=67101]: Got next segment. Inline: false Dec 8 20:13:16.313623854 (76h44m16.184706889s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 20:13:16.313629435 (76h44m16.18471247s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 20:13:18.037990476 (76h44m17.909073521s) [seg_pos={0 0}, upload=67101]: Begin upload segment... Dec 8 20:13:18.038044659 (76h44m17.909127694s) [seg_pos={0 0}, upload=67101]: Done begin upload segment. Dec 8 20:13:18.038075859 (76h44m17.909158914s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038191428 (76h44m17.909274463s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038908023 (76h44m17.909991068s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039373890 (76h44m17.910456925s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039816273 (76h44m17.910899318s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040292149 (76h44m17.911375194s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040745432 (76h44m17.911828477s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041266104 (76h44m17.912349139s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041787887 (76h44m17.912870952s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.042946814 (76h44m17.914029849s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.044580184 (76h44m17.915663219s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.047199231 (76h44m17.918282266s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.049620632 (76h44m17.920703677s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.051625078 (76h44m17.922708143s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.053186752 (76h44m17.924269787s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.054291115 (76h44m17.92537415s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:19.111879942 (76h44m18.982962987s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: storage node overloaded, request limit: 2000 Dec 8 20:13:19.643615670 (76h44m19.514698706s) [upload=67101]: Next returned nil segment Dec 8 20:13:19.643629617 (76h44m19.514712652s) [upload=67101]: Waiting for error group managing segments... Dec 8 20:13:20.026246645 (76h44m19.89732969s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.027742514 (76h44m19.898825559s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.028492581 (76h44m19.899575616s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029004767 (76h44m19.900087802s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029438392 (76h44m19.900521427s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.031410738 (76h44m19.902493783s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.034141027 (76h44m19.905224072s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.037360508 (76h44m19.908443543s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.038059108 (76h44m19.909142153s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053205520 (76h44m19.924288565s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053898059 (76h44m19.924981114s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.054182971 (76h44m19.925266006s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058658112 (76h44m19.929741157s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058679072 (76h44m19.929762107s) [seg_pos={0 0}, upload=67101]: Segment reached optimal threshold of 13 pieces. Dec 8 20:13:20.058873853 (76h44m19.929956888s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.058880526 (76h44m19.929963561s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059077551 (76h44m19.930160586s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.059093532 (76h44m19.930176567s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059116485 (76h44m19.93019953s) [seg_pos={0 0}, upload=67101]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 8 20:13:20.059191879 (76h44m19.930274924s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:13:20.059196798 (76h44m19.930279833s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:19:41.003042505 (76h50m40.87412554s) [upload=67101]: Done uploading segments... Dec 8 20:19:41.003155129 (76h50m40.874238175s) [upload=67101]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.628038596s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [........X.................................X.................] duration: 476.593951ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:22:14 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 476.642544ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..........X...................................X.............] duration: 13.214042764s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:22:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 13.214068693s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.........X.................................X................] duration: 479.916075ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:21:39 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 479.943907ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.........X.............] duration: 6.063527456s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:21:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.063554258s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.730413647s trace: https://jaeger.dev.storj.io/trace/690c0830f44a9942 FAIL (2025-12-08 20:21:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 8 20:13:16.312406346 (76h44m16.183489391s) [upload=67101]: Starting upload Dec 8 20:13:16.312474916 (76h44m16.183557952s) [upload=67101]: Done starting upload Dec 8 20:13:16.312661411 (76h44m16.183744446s) [upload=67101]: Uploading segments... Dec 8 20:13:16.313579429 (76h44m16.184662474s) [upload=67101]: Got next segment. Inline: false Dec 8 20:13:16.313623854 (76h44m16.184706889s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 20:13:16.313629435 (76h44m16.18471247s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 20:13:18.037990476 (76h44m17.909073521s) [seg_pos={0 0}, upload=67101]: Begin upload segment... Dec 8 20:13:18.038044659 (76h44m17.909127694s) [seg_pos={0 0}, upload=67101]: Done begin upload segment. Dec 8 20:13:18.038075859 (76h44m17.909158914s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038191428 (76h44m17.909274463s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038908023 (76h44m17.909991068s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039373890 (76h44m17.910456925s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039816273 (76h44m17.910899318s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040292149 (76h44m17.911375194s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040745432 (76h44m17.911828477s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041266104 (76h44m17.912349139s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041787887 (76h44m17.912870952s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.042946814 (76h44m17.914029849s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.044580184 (76h44m17.915663219s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.047199231 (76h44m17.918282266s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.049620632 (76h44m17.920703677s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.051625078 (76h44m17.922708143s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.053186752 (76h44m17.924269787s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.054291115 (76h44m17.92537415s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:19.111879942 (76h44m18.982962987s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: storage node overloaded, request limit: 2000 Dec 8 20:13:19.643615670 (76h44m19.514698706s) [upload=67101]: Next returned nil segment Dec 8 20:13:19.643629617 (76h44m19.514712652s) [upload=67101]: Waiting for error group managing segments... Dec 8 20:13:20.026246645 (76h44m19.89732969s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.027742514 (76h44m19.898825559s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.028492581 (76h44m19.899575616s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029004767 (76h44m19.900087802s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029438392 (76h44m19.900521427s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.031410738 (76h44m19.902493783s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.034141027 (76h44m19.905224072s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.037360508 (76h44m19.908443543s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.038059108 (76h44m19.909142153s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053205520 (76h44m19.924288565s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053898059 (76h44m19.924981114s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.054182971 (76h44m19.925266006s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058658112 (76h44m19.929741157s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058679072 (76h44m19.929762107s) [seg_pos={0 0}, upload=67101]: Segment reached optimal threshold of 13 pieces. Dec 8 20:13:20.058873853 (76h44m19.929956888s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.058880526 (76h44m19.929963561s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059077551 (76h44m19.930160586s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.059093532 (76h44m19.930176567s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059116485 (76h44m19.93019953s) [seg_pos={0 0}, upload=67101]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 8 20:13:20.059191879 (76h44m19.930274924s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:13:20.059196798 (76h44m19.930279833s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:19:41.003042505 (76h50m40.87412554s) [upload=67101]: Done uploading segments... Dec 8 20:19:41.003155129 (76h50m40.874238175s) [upload=67101]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.730453242s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......X...................X.......................X....X....] duration: 5.752814065s trace: https://jaeger.dev.storj.io/trace/cbe2f3dd02a455b FAIL (2025-12-08 20:21:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.752831528s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..........X.................................X...............] duration: 468.765852ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:21:04 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 468.807511ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...........X...................................X............] duration: 4.613819938s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:20:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.613848832s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.........X............] duration: 5.709498144s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:20:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.709527137s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.680230472s trace: https://jaeger.dev.storj.io/trace/690c0830f44a9942 FAIL (2025-12-08 20:20:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 8 20:13:16.312406346 (76h44m16.183489391s) [upload=67101]: Starting upload Dec 8 20:13:16.312474916 (76h44m16.183557952s) [upload=67101]: Done starting upload Dec 8 20:13:16.312661411 (76h44m16.183744446s) [upload=67101]: Uploading segments... Dec 8 20:13:16.313579429 (76h44m16.184662474s) [upload=67101]: Got next segment. Inline: false Dec 8 20:13:16.313623854 (76h44m16.184706889s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 20:13:16.313629435 (76h44m16.18471247s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 20:13:18.037990476 (76h44m17.909073521s) [seg_pos={0 0}, upload=67101]: Begin upload segment... Dec 8 20:13:18.038044659 (76h44m17.909127694s) [seg_pos={0 0}, upload=67101]: Done begin upload segment. Dec 8 20:13:18.038075859 (76h44m17.909158914s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038191428 (76h44m17.909274463s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038908023 (76h44m17.909991068s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039373890 (76h44m17.910456925s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039816273 (76h44m17.910899318s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040292149 (76h44m17.911375194s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040745432 (76h44m17.911828477s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041266104 (76h44m17.912349139s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041787887 (76h44m17.912870952s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.042946814 (76h44m17.914029849s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.044580184 (76h44m17.915663219s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.047199231 (76h44m17.918282266s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.049620632 (76h44m17.920703677s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.051625078 (76h44m17.922708143s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.053186752 (76h44m17.924269787s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.054291115 (76h44m17.92537415s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:19.111879942 (76h44m18.982962987s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: storage node overloaded, request limit: 2000 Dec 8 20:13:19.643615670 (76h44m19.514698706s) [upload=67101]: Next returned nil segment Dec 8 20:13:19.643629617 (76h44m19.514712652s) [upload=67101]: Waiting for error group managing segments... Dec 8 20:13:20.026246645 (76h44m19.89732969s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.027742514 (76h44m19.898825559s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.028492581 (76h44m19.899575616s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029004767 (76h44m19.900087802s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029438392 (76h44m19.900521427s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.031410738 (76h44m19.902493783s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.034141027 (76h44m19.905224072s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.037360508 (76h44m19.908443543s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.038059108 (76h44m19.909142153s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053205520 (76h44m19.924288565s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053898059 (76h44m19.924981114s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.054182971 (76h44m19.925266006s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058658112 (76h44m19.929741157s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058679072 (76h44m19.929762107s) [seg_pos={0 0}, upload=67101]: Segment reached optimal threshold of 13 pieces. Dec 8 20:13:20.058873853 (76h44m19.929956888s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.058880526 (76h44m19.929963561s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059077551 (76h44m19.930160586s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.059093532 (76h44m19.930176567s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059116485 (76h44m19.93019953s) [seg_pos={0 0}, upload=67101]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 8 20:13:20.059191879 (76h44m19.930274924s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:13:20.059196798 (76h44m19.930279833s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:19:41.003042505 (76h50m40.87412554s) [upload=67101]: Done uploading segments... Dec 8 20:19:41.003155129 (76h50m40.874238175s) [upload=67101]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.680258337s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............X...................................X...........] duration: 4.675919937s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:20:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.675961496s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...........X.................................X..............] duration: 472.126632ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:20:15 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 472.156287ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.........X...........] duration: 5.957597542s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:20:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.957670922s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......X...................X.......................X....X...] duration: 5.999123175s trace: https://jaeger.dev.storj.io/trace/cbe2f3dd02a455b FAIL (2025-12-08 20:20:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.999155908s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.714585347s trace: https://jaeger.dev.storj.io/trace/690c0830f44a9942 FAIL (2025-12-08 20:19:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 8 20:13:16.312406346 (76h44m16.183489391s) [upload=67101]: Starting upload Dec 8 20:13:16.312474916 (76h44m16.183557952s) [upload=67101]: Done starting upload Dec 8 20:13:16.312661411 (76h44m16.183744446s) [upload=67101]: Uploading segments... Dec 8 20:13:16.313579429 (76h44m16.184662474s) [upload=67101]: Got next segment. Inline: false Dec 8 20:13:16.313623854 (76h44m16.184706889s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 20:13:16.313629435 (76h44m16.18471247s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 20:13:18.037990476 (76h44m17.909073521s) [seg_pos={0 0}, upload=67101]: Begin upload segment... Dec 8 20:13:18.038044659 (76h44m17.909127694s) [seg_pos={0 0}, upload=67101]: Done begin upload segment. Dec 8 20:13:18.038075859 (76h44m17.909158914s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038191428 (76h44m17.909274463s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038908023 (76h44m17.909991068s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039373890 (76h44m17.910456925s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039816273 (76h44m17.910899318s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040292149 (76h44m17.911375194s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040745432 (76h44m17.911828477s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041266104 (76h44m17.912349139s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041787887 (76h44m17.912870952s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.042946814 (76h44m17.914029849s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.044580184 (76h44m17.915663219s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.047199231 (76h44m17.918282266s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.049620632 (76h44m17.920703677s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.051625078 (76h44m17.922708143s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.053186752 (76h44m17.924269787s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.054291115 (76h44m17.92537415s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:19.111879942 (76h44m18.982962987s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: storage node overloaded, request limit: 2000 Dec 8 20:13:19.643615670 (76h44m19.514698706s) [upload=67101]: Next returned nil segment Dec 8 20:13:19.643629617 (76h44m19.514712652s) [upload=67101]: Waiting for error group managing segments... Dec 8 20:13:20.026246645 (76h44m19.89732969s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.027742514 (76h44m19.898825559s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.028492581 (76h44m19.899575616s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029004767 (76h44m19.900087802s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029438392 (76h44m19.900521427s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.031410738 (76h44m19.902493783s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.034141027 (76h44m19.905224072s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.037360508 (76h44m19.908443543s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.038059108 (76h44m19.909142153s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053205520 (76h44m19.924288565s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053898059 (76h44m19.924981114s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.054182971 (76h44m19.925266006s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058658112 (76h44m19.929741157s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058679072 (76h44m19.929762107s) [seg_pos={0 0}, upload=67101]: Segment reached optimal threshold of 13 pieces. Dec 8 20:13:20.058873853 (76h44m19.929956888s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.058880526 (76h44m19.929963561s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059077551 (76h44m19.930160586s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.059093532 (76h44m19.930176567s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059116485 (76h44m19.93019953s) [seg_pos={0 0}, upload=67101]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 8 20:13:20.059191879 (76h44m19.930274924s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:13:20.059196798 (76h44m19.930279833s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:19:41.003042505 (76h50m40.87412554s) [upload=67101]: Done uploading segments... Dec 8 20:19:41.003155129 (76h50m40.874238175s) [upload=67101]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.71460341s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [............X.................................X.............] duration: 472.223426ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:19:44 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 472.252192ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............X...................................X..........] duration: 5.061021068s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:19:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.061049703s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 6m26.703815156s trace: https://jaeger.dev.storj.io/trace/690c0830f44a9942 FAIL (2025-12-08 20:19:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 8 20:13:16.312406346 (76h44m16.183489391s) [upload=67101]: Starting upload Dec 8 20:13:16.312474916 (76h44m16.183557952s) [upload=67101]: Done starting upload Dec 8 20:13:16.312661411 (76h44m16.183744446s) [upload=67101]: Uploading segments... Dec 8 20:13:16.313579429 (76h44m16.184662474s) [upload=67101]: Got next segment. Inline: false Dec 8 20:13:16.313623854 (76h44m16.184706889s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 20:13:16.313629435 (76h44m16.18471247s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 20:13:18.037990476 (76h44m17.909073521s) [seg_pos={0 0}, upload=67101]: Begin upload segment... Dec 8 20:13:18.038044659 (76h44m17.909127694s) [seg_pos={0 0}, upload=67101]: Done begin upload segment. Dec 8 20:13:18.038075859 (76h44m17.909158914s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038191428 (76h44m17.909274463s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.038908023 (76h44m17.909991068s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039373890 (76h44m17.910456925s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.039816273 (76h44m17.910899318s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040292149 (76h44m17.911375194s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.040745432 (76h44m17.911828477s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041266104 (76h44m17.912349139s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.041787887 (76h44m17.912870952s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.042946814 (76h44m17.914029849s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.044580184 (76h44m17.915663219s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.047199231 (76h44m17.918282266s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.049620632 (76h44m17.920703677s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.051625078 (76h44m17.922708143s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.053186752 (76h44m17.924269787s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:18.054291115 (76h44m17.92537415s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Uploading piece... Dec 8 20:13:19.111879942 (76h44m18.982962987s) [noise=true , address=77.90.31.10:4517 , piece_id=BMXTXAXNYZ5U5T54U57PMGP4KZO332K2VEP4KO5NEYC7TZ2S53SA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: storage node overloaded, request limit: 2000 Dec 8 20:13:19.643615670 (76h44m19.514698706s) [upload=67101]: Next returned nil segment Dec 8 20:13:19.643629617 (76h44m19.514712652s) [upload=67101]: Waiting for error group managing segments... Dec 8 20:13:20.026246645 (76h44m19.89732969s) [noise=true , address=145.79.227.32:4834 , piece_id=WDZ3N3A3CNPAECT7HW2VZ2K7JEJVOZJCZR3BKHRBVR3XHGPGTKPQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.027742514 (76h44m19.898825559s) [noise=true , address=145.79.227.25:4619 , piece_id=5M5PTNON4YZ7INE34X7I6BIJVQF543VQWALEZBCZEEEQ26UMGYQA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.028492581 (76h44m19.899575616s) [noise=true , address=145.79.227.41:4085 , piece_id=FFFFOS7PH3SN3S4NWJJRYP772D725TSGCBKXIZ7FTEDBPO3TNR3Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029004767 (76h44m19.900087802s) [noise=true , address=145.79.227.16:4912 , piece_id=J7BVXTHOSEPXKDDWXPV7JIUWW5NNWLETI7S4IOVPDG6ZUVPATOXQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.029438392 (76h44m19.900521427s) [noise=true , address=145.79.227.24:4583 , piece_id=MTNWT2AOTPYXLNVPSOKK5ZOAY7U3UH376V232TZKE56UM2CMDVAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.031410738 (76h44m19.902493783s) [noise=true , address=145.79.227.42:4121 , piece_id=ICWYGS3AGWC47JS4PUC4GHT2YOV5UTD5RYL73JFTWZRNEMAGMRZA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.034141027 (76h44m19.905224072s) [noise=true , address=145.79.227.31:4799 , piece_id=RNPR7HC6YANTTQW7STMNTHYANE7ZD5XC2N5DMLQQ5YBULNUSFK7A, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.037360508 (76h44m19.908443543s) [noise=true , address=145.79.227.22:4511 , piece_id=CFUDAP3RXMYJCS5GQJVYCKXTN6RLXG7NSQH6EHTZPSYCXPSPBEFQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.038059108 (76h44m19.909142153s) [noise=true , address=145.79.227.39:4015 , piece_id=UX4HGNQIHC3FCNGUGIAZYPNYYTLNXMYIND7H7PZ6DCD5EXS67Y7Q, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053205520 (76h44m19.924288565s) [noise=true , address=145.79.227.16:4910 , piece_id=YIHYKALCHWYGMAUKS3RPUQ2WOAQKCRQV6GZPFM27K37CKTTJSDPA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.053898059 (76h44m19.924981114s) [noise=true , address=145.79.227.20:4437 , piece_id=G76GCXUI5YH3CMQV47V6FSFDDSOBIIFMS24UJVTNXRR2MM4UZ5ZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.054182971 (76h44m19.925266006s) [noise=true , address=145.79.227.27:4691 , piece_id=R5VVIJNPYZIDQW5WSJMEYS4D55AHXXMACQY5GM5IOJXVKNOGBEAQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058658112 (76h44m19.929741157s) [noise=true , address=145.79.227.29:4942 , piece_id=ZYT75AZ7X56NBH3OVLWL27J5FRRN4WKV5VEIKEXN32E6P5OQNOZQ, seg_pos={0 0}, upload=67101]: Done uploading piece. err: <nil> Dec 8 20:13:20.058679072 (76h44m19.929762107s) [seg_pos={0 0}, upload=67101]: Segment reached optimal threshold of 13 pieces. Dec 8 20:13:20.058873853 (76h44m19.929956888s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.058880526 (76h44m19.929963561s) [noise=true , address=145.79.227.31:4797 , piece_id=IHEVQ2JUPASTR3CU7QMNAQVSYQ65LEV5SR2KO4GCEOKAKQR5Y5NA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059077551 (76h44m19.930160586s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Dec 8 20:13:20.059093532 (76h44m19.930176567s) [noise=true , address=145.79.227.20:4438 , piece_id=QZQULVSGCAOTQEE5MBUONJYAJIO3GPUNKB4CEVMK3NA2J3BYZ6GA, seg_pos={0 0}, upload=67101]: Long tail cancellation due to optimal threshold Dec 8 20:13:20.059116485 (76h44m19.93019953s) [seg_pos={0 0}, upload=67101]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 8 20:13:20.059191879 (76h44m19.930274924s) [upload=67101]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:13:20.059196798 (76h44m19.930279833s) [upload=67101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:19:41.003042505 (76h50m40.87412554s) [upload=67101]: Done uploading segments... Dec 8 20:19:41.003155129 (76h50m40.874238175s) [upload=67101]: Upload finished. err: metaclient: context deadline exceeded Duration: 6m26.703839873s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.........X..........] duration: 5.785604873s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:19:30 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.785629108s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.............X.................................X............] duration: 468.061381ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:19:28 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 468.088493ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........X...................X.......................X....X..] duration: 5.702197539s trace: https://jaeger.dev.storj.io/trace/cbe2f3dd02a455b FAIL (2025-12-08 20:19:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.70222967s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..............X.................................X...........] duration: 467.288017ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:19:12 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 467.313887ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.........X...................X.......................X....X.] duration: 5.953005808s trace: https://jaeger.dev.storj.io/trace/cbe2f3dd02a455b FAIL (2025-12-08 20:18:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.953018953s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..........X...................X.......................X....X] duration: 1m0.000840754s trace: https://jaeger.dev.storj.io/trace/cbe2f3dd02a455b FAIL (2025-12-08 20:18:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000864189s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...............X.................................X..........] duration: 474.059103ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:18:44 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 474.08971ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............X...................................X.........] duration: 5.152447539s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:18:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.152476184s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.........X.........] duration: 6.013105318s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:18:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.013135816s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [................X.................................X.........] duration: 463.377813ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:18:14 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 463.408552ms core satellite eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.................X.................................X........] duration: 477.351901ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:17:54 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 477.376669ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...............X...................................X........] duration: 5.800326529s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:17:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.800358038s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.........X........] duration: 6.004621053s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:17:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.004649488s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.017881913s trace: https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9 FAIL (2025-12-08 20:17:38 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.017910117s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........X...................X.......................X....] duration: 5.857266713s trace: https://jaeger.dev.storj.io/trace/50d717f68b7ecd80 FAIL (2025-12-08 20:17:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.857290397s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [................X...................................X.......] duration: 5.644203728s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:17:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.644235679s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.........X.......] duration: 5.987232292s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:16:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.987257479s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..................X.................................X.......] duration: 470.610274ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:16:49 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 470.693442ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.........X......] duration: 5.970590814s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:16:42 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.970628255s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.................X...................................X......] duration: 4.643941854s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:16:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.643968866s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [X...........X...................X.......................X...] duration: 5.863464113s trace: https://jaeger.dev.storj.io/trace/50d717f68b7ecd80 FAIL (2025-12-08 20:16:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.863490532s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.995316704s trace: https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9 FAIL (2025-12-08 20:16:32 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.995344578s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..................X...................................X.....] duration: 5.269831873s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:16:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.269864335s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...................X.................................X......] duration: 470.446492ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4b87d2bbd73f899c FAIL (2025-12-08 20:15:45 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 470.493511ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.........X.....] duration: 5.834187766s trace: https://jaeger.dev.storj.io/trace/6fb9d317d1dcf919 FAIL (2025-12-08 20:15:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.834214687s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.052259035s trace: https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9 FAIL (2025-12-08 20:15:42 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.052281637s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...................X...................................X....] duration: 5.667002041s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:15:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.667026336s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.X...........X...................X.......................X..] duration: 5.86153023s trace: https://jaeger.dev.storj.io/trace/50d717f68b7ecd80 FAIL (2025-12-08 20:15:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.861557732s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....................X...................................X...] duration: 5.14969139s trace: https://jaeger.dev.storj.io/trace/5fa91278c47e99e6 FAIL (2025-12-08 20:14:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.542756499 (76h43m43.413839544s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543240850 (76h43m43.414323885s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.543683303 (76h43m43.414766338s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.544210737 (76h43m43.415293772s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.545783021 (76h43m43.416866056s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.548444850 (76h43m43.419527895s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.550084182 (76h43m43.421167217s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.551490920 (76h43m43.422573955s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.552901015 (76h43m43.42398405s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:43.554528074 (76h43m43.425611109s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Uploading piece... Dec 8 20:12:44.367019115 (76h43m44.23810216s) [upload=67092]: Next returned nil segment Dec 8 20:12:44.367036829 (76h43m44.238119874s) [upload=67092]: Waiting for error group managing segments... Dec 8 20:12:44.757695779 (76h43m44.628778814s) [noise=true , address=185.167.204.206:4060 , piece_id=ND5G6F745HRT4H325ZJL7Z5DWDJ4W4WV73HCZ4ERCRZS6B25VIWQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.762182402 (76h43m44.633265447s) [noise=true , address=185.167.204.205:4004 , piece_id=TXPSWSQQBLLOZYM5M7OM22HUQ6KEUJOLUMVDAG23GURRVEUVAWUQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.763337131 (76h43m44.634420166s) [noise=true , address=185.167.204.205:4035 , piece_id=6TCDNDF5XDWLBJH7CQSNKUJNBUWP5NIJXVKVSNMZTB7J56ZN2DDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.764976814 (76h43m44.636059849s) [noise=true , address=185.167.204.205:4016 , piece_id=6R2V7UAVOQISGE4HJWTJUKZZJC4AQOWFDOM2WNIQRFMPXDMYCMPA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.767445675 (76h43m44.63852872s) [noise=true , address=185.167.204.206:4071 , piece_id=UQ6DRVVVA2CNIAX45IVQLIKEVK2X73QKBMSKAICT57ZFSHBEY5RQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.768606505 (76h43m44.63968955s) [noise=true , address=185.167.204.206:4048 , piece_id=QWOPWDZLNZA2USWC2GITYCOQSFMOVLYS5DVPE7INVKH7QC3XNX4Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771330402 (76h43m44.642413437s) [noise=true , address=185.167.204.206:4056 , piece_id=H25V5UWRVPRIHGTT7DLLSBL3SEVYMZ4PM77WQIFF54WAJAYLF6WQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.771936476 (76h43m44.643019511s) [noise=true , address=185.167.204.206:4072 , piece_id=F5N6SYF37XDCQYV54I276ISILTBNPWLN2BTNUCJTM4242VPEHZ4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772397935 (76h43m44.64348097s) [noise=true , address=185.167.204.205:4022 , piece_id=FGZCGGNXKE4DXG23JY22PNGCZVH5COVGI4LBBGEGB2VT5WTTSG7A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.772821992 (76h43m44.643905027s) [noise=true , address=185.167.204.206:4070 , piece_id=FC5STX6BY57UXDZIYJRM4TZTKLQ2WF5QU253M372NETRRBS74DBQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773215182 (76h43m44.644298217s) [noise=true , address=185.167.204.206:4065 , piece_id=K6Z5AOIHHE2KHUWCBFECRASMZLKFFNT2XE7CQC2OAOZJDDRDGJXQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.773618198 (76h43m44.644701233s) [noise=true , address=185.167.204.206:4057 , piece_id=MBY5BDOP5VQQ36JQUGE2MF3Z3KBIG4YWZEG573II24XF6YZBUPRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.778169605 (76h43m44.64925264s) [noise=true , address=185.167.204.205:4026 , piece_id=DDC35FH3APO64ELR4S3XX646VYFEDEA3VLGDJ57PVCP37STLR3TA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.779154380 (76h43m44.650237415s) [noise=true , address=185.167.204.205:4006 , piece_id=IQYRNR6TZ6DW6YTM7UDNGIRS45P4FMNDKPZLEE2ES2GM7K4NUADQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780154986 (76h43m44.651238021s) [noise=true , address=185.167.204.206:4038 , piece_id=VRRCFONWGGK2TSTD5O36CC3MVPSCV7VYX7FEDEY4P3CA7NU4HMLQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.780715603 (76h43m44.651798638s) [noise=true , address=185.167.204.206:4068 , piece_id=WC62ZMRXXDD6R2AE4YWEU6EQLDDNUNHUJQTO7LQ7FPNBM3VMXNDQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.782187295 (76h43m44.65327033s) [noise=true , address=185.167.204.205:4033 , piece_id=N3YACMETXALAPTCCO7FVM4R7IJ3PT4LINSGOLJHD75M6ASEQLH3Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.783292590 (76h43m44.654375625s) [noise=true , address=185.167.204.205:4014 , piece_id=H3TIJ5LH4S4NWJZCG7LROYHAHBYLSSV6W7CVWTOI73LPAERQXKIQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784377676 (76h43m44.655460711s) [noise=true , address=185.167.204.205:4010 , piece_id=VITSWV5I53QHI5BO6TPRCB4M5JH3YDLCVDW5CRDC5KVHKRYWLI6Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:44.784629415 (76h43m44.65571246s) [noise=true , address=185.167.204.205:4019 , piece_id=PQADAZMKMCRE5H62DRBANZQGTNBWWFOVGQC4EQBJ6ZXFFL2WUA4A, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.328109727 (76h43m45.199192762s) [noise=true , address=81.171.14.70:4033 , piece_id=CXX5EV33TA5EOQUDLJCA5GFQBBLGYNT4FS6C6PEFVHPKXLID2MRA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.356147476 (76h43m45.227230512s) [noise=true , address=81.171.14.70:4042 , piece_id=M36QWUV6JSKWL5VIXBB3FQ7UQ6F6K5S5ZLXVXI2ZF5F4QPTV72SQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.399467146 (76h43m45.270550181s) [noise=true , address=81.171.14.70:4028 , piece_id=QRYUY4YLMXX7G7AHEOWGHPZKG3KHTCRBQ5S6DN4Q4V5W6EQDRN5Q, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.861490429 (76h43m45.732573474s) [noise=true , address=81.171.14.70:4048 , piece_id=VLOTEN2GWXJTAEBTZLFH2HUVRTQCI4WCSNVYOVNJNMM6X5AEV6NA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:45.975706748 (76h43m45.846789793s) [noise=true , address=89.149.201.147:4011 , piece_id=ZPQ35LPQV7BEIHHLU6J4NVHOFRE7ZMEMBQUYLQVNMENJ3VMCDWEA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026462388 (76h43m45.897545433s) [noise=true , address=89.149.201.147:4009 , piece_id=RGKFECPRTUWYTZC4DB6ASMPVNWCB7JOHWGJBVOGJMFQCO36MATKQ, seg_pos={0 0}, upload=67092]: Done uploading piece. err: <nil> Dec 8 20:12:46.026486353 (76h43m45.897569398s) [seg_pos={0 0}, upload=67092]: Segment reached optimal threshold of 26 pieces. Dec 8 20:12:46.026747771 (76h43m45.897830826s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.026774823 (76h43m45.897857868s) [noise=true , address=89.149.201.147:4004 , piece_id=UONP7JUXGL35ZRI64K63PCEPXAVSXDJRSXBG56JC34GGR7NHMBIA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027019298 (76h43m45.898102333s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn): protocol: expected piece hash; context canceled; EOF Dec 8 20:12:46.027036521 (76h43m45.898119556s) [noise=true , address=89.149.201.147:4008 , piece_id=QBJZ4ONOAJNB3ST2443IBDJFWYJV27QUD7BFSY2IPI6E5FQ6WSJA, seg_pos={0 0}, upload=67092]: Long tail cancellation due to optimal threshold Dec 8 20:12:46.027063101 (76h43m45.898146146s) [seg_pos={0 0}, upload=67092]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 20:12:46.027209430 (76h43m45.898292465s) [upload=67092]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 20:12:46.027214870 (76h43m45.898297915s) [upload=67092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 20:13:42.864213034 (76h44m42.735296079s) [upload=67092]: Done uploading segments... Dec 8 20:13:42.864305480 (76h44m42.735388515s) [upload=67092]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.149721908s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.99205302s trace: https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9 FAIL (2025-12-08 20:14:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.992104317s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)