Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.792655808s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:50:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.792670676s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 4.883586157s trace: https://jaeger.dev.storj.io/trace/456cfc131726c52f FAIL (2025-12-08 03:50:47 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.883619159s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [X.........X...............X...................X.............] duration: 5.795487011s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:50:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.795512518s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 710.601563ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6efc88c1a2a7b7f2 FAIL (2025-12-08 03:50:46 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 710.628793ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...X...X.X...........X..........X......X....X...........X...] duration: 3.444215406s trace: https://jaeger.dev.storj.io/trace/59dc391c761605b5 FAIL (2025-12-08 03:50:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.444244251s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X...............X..........................] duration: 5.656957146s trace: https://jaeger.dev.storj.io/trace/15c53885b53c2c0b FAIL (2025-12-08 03:50:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.65698513s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.764542683s trace: https://jaeger.dev.storj.io/trace/5ce16d59ac7ca3 FAIL (2025-12-08 03:50:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== V3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478633206 (59h44m15.349716251s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478935492 (59h44m15.350018537s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479212750 (59h44m15.350295785s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479658087 (59h44m15.350741132s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.480729167 (59h44m15.351812212s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481358675 (59h44m15.35244171s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481870881 (59h44m15.352953916s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482011018 (59h44m15.353094063s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482599078 (59h44m15.353682113s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.483043735 (59h44m15.35412679s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:16.296121260 (59h44m16.167204295s) [upload=52461]: Next returned nil segment Dec 8 03:13:16.296136789 (59h44m16.167219824s) [upload=52461]: Waiting for error group managing segments... Dec 8 03:13:16.688782787 (59h44m16.559865822s) [noise=true , address=185.167.204.206:4059 , piece_id=G4AB6LYXSDKJOYKXNAILU34CEGNPXWYJQICBGHLDTWNH3TMHA7JA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.689199911 (59h44m16.560282946s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690209955 (59h44m16.561293s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690460382 (59h44m16.561543427s) [noise=true , address=185.167.204.205:4010 , piece_id=5TOGC4H52AGJS6ZZJJM56DZXZQJ2FBK2YZO6634ACFVLFGOOHC5A, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.696363612 (59h44m16.567446657s) [noise=true , address=185.167.204.206:4051 , piece_id=SPX6V5WZAWUFRZ23KZKQPPWS3VX37O6EOOQTCDRXXNMIGQGSERMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697250752 (59h44m16.568333787s) [noise=true , address=185.167.204.205:4011 , piece_id=ZM5KFXNZTZTU6WS33BQ4HHO2G4EHQGNCZKVNSKXPXEXAM4RW6BQA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697962948 (59h44m16.569045983s) [noise=true , address=185.167.204.206:4072 , piece_id=F5EN43FGL2LXIZU4KHKDGCXM3O7YJM37AT2XBHRLSEOWDJMP5GCA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.700404106 (59h44m16.571487151s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701120280 (59h44m16.572203315s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701632446 (59h44m16.572715481s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.702967407 (59h44m16.574050453s) [noise=true , address=185.167.204.205:4019 , piece_id=URV3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.706243235 (59h44m16.57732628s) [noise=true , address=185.167.204.205:4035 , piece_id=6NZETOEX253U2AGIILFBIK3JGBL2S7PJV6WE77ITLVEROV6SZVWA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.710948004 (59h44m16.582031049s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.712808888 (59h44m16.583891933s) [noise=true , address=185.167.204.206:4065 , piece_id=EPW6FRXQZKBOVMTROQYK5O33BBZHIW7BU5HMHKPMOWZNICQJWWMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.193700893 (59h44m17.064783928s) [noise=true , address=81.171.14.70:4031 , piece_id=KPTJZ4FYTXEYVVSLRQD42OL7NGOTXQF5RVPA3PTC4L7W5AZWQNMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.205494490 (59h44m17.076577535s) [noise=true , address=81.171.14.70:4026 , piece_id=UQBSQNBMHZY7733ZI6VNDEU5WXZ2GUBRWPUL3W5LRLWEVLXFN7GQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.213382981 (59h44m17.084466026s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.223902893 (59h44m17.094985938s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.272328052 (59h44m17.143411088s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.314279466 (59h44m17.185362501s) [noise=true , address=81.171.14.70:4038 , piece_id=QWZFMCB3BKGSROA6WTKBMTIKIKZ744UZ7222NGF2ZRMXMJH55K2Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.339184389 (59h44m17.210267434s) [noise=true , address=81.171.14.70:4029 , piece_id=N2MJLXIIMGZZG3TCJ5LJUGH4J5H5OXA6GOGLSBTN6BEMDXOWGTDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.352328829 (59h44m17.223411864s) [noise=true , address=81.171.14.70:4028 , piece_id=K4GHMGJRVY5233RXSDQSHW4OQSXK6A534J36KNIJZNRBLE2YMNOQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.363261126 (59h44m17.234344171s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.770690916 (59h44m17.641773951s) [noise=true , address=81.171.14.70:4027 , piece_id=SRP3XCTIO67QDY2APAFNUGDVJYP4NTNH2ALVZO3CKQWJRGONSDHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.980125476 (59h44m17.851208521s) [noise=true , address=89.149.201.147:4005 , piece_id=DLFODTUO7RTGVF5CKBB5NFBCRLEVBR57I2W746O6JQX4ZJ6Q4RUA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040367989 (59h44m17.911451024s) [noise=true , address=89.149.201.147:4007 , piece_id=33HGXVE4ZG2EH6ROJSXS2WRSZSM3XFH7JXLTHT6QOIADMWWMB4SA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040388418 (59h44m17.911471453s) [seg_pos={0 0}, upload=52461]: Segment reached optimal threshold of 26 pieces. Dec 8 03:13:18.040618335 (59h44m17.91170137s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040631010 (59h44m17.911714055s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040763592 (59h44m17.911846647s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040782438 (59h44m17.911865483s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040832964 (59h44m17.911916009s) [seg_pos={0 0}, upload=52461]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 03:13:18.040964104 (59h44m17.912047149s) [upload=52461]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:13:18.040993821 (59h44m17.912076866s) [upload=52461]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:25:31.019642569 (59h56m30.890725614s) [upload=52461]: Done uploading segments... Dec 8 03:25:31.019743331 (59h56m30.890826366s) [upload=52461]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.764576026s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.808628444s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:50:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.808661247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................X......X..............................] duration: 2.565201529s trace: https://jaeger.dev.storj.io/trace/67981d3422e72759 FAIL (2025-12-08 03:50:11 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.565231084s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 822.943785ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6efc88c1a2a7b7f2 FAIL (2025-12-08 03:50:06 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 822.97293ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.735466652s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:49:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.735495096s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.X.........X...............X...................X............] duration: 5.804334255s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:49:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.804363793s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X...............X.........................] duration: 5.96108339s trace: https://jaeger.dev.storj.io/trace/15c53885b53c2c0b FAIL (2025-12-08 03:49:53 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.961106705s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X......X.............................] duration: 2.040281247s trace: https://jaeger.dev.storj.io/trace/67981d3422e72759 FAIL (2025-12-08 03:49:45 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.040315111s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.109886166s trace: https://jaeger.dev.storj.io/trace/5ce16d59ac7ca3 FAIL (2025-12-08 03:49:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== V3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478633206 (59h44m15.349716251s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478935492 (59h44m15.350018537s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479212750 (59h44m15.350295785s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479658087 (59h44m15.350741132s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.480729167 (59h44m15.351812212s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481358675 (59h44m15.35244171s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481870881 (59h44m15.352953916s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482011018 (59h44m15.353094063s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482599078 (59h44m15.353682113s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.483043735 (59h44m15.35412679s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:16.296121260 (59h44m16.167204295s) [upload=52461]: Next returned nil segment Dec 8 03:13:16.296136789 (59h44m16.167219824s) [upload=52461]: Waiting for error group managing segments... Dec 8 03:13:16.688782787 (59h44m16.559865822s) [noise=true , address=185.167.204.206:4059 , piece_id=G4AB6LYXSDKJOYKXNAILU34CEGNPXWYJQICBGHLDTWNH3TMHA7JA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.689199911 (59h44m16.560282946s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690209955 (59h44m16.561293s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690460382 (59h44m16.561543427s) [noise=true , address=185.167.204.205:4010 , piece_id=5TOGC4H52AGJS6ZZJJM56DZXZQJ2FBK2YZO6634ACFVLFGOOHC5A, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.696363612 (59h44m16.567446657s) [noise=true , address=185.167.204.206:4051 , piece_id=SPX6V5WZAWUFRZ23KZKQPPWS3VX37O6EOOQTCDRXXNMIGQGSERMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697250752 (59h44m16.568333787s) [noise=true , address=185.167.204.205:4011 , piece_id=ZM5KFXNZTZTU6WS33BQ4HHO2G4EHQGNCZKVNSKXPXEXAM4RW6BQA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697962948 (59h44m16.569045983s) [noise=true , address=185.167.204.206:4072 , piece_id=F5EN43FGL2LXIZU4KHKDGCXM3O7YJM37AT2XBHRLSEOWDJMP5GCA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.700404106 (59h44m16.571487151s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701120280 (59h44m16.572203315s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701632446 (59h44m16.572715481s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.702967407 (59h44m16.574050453s) [noise=true , address=185.167.204.205:4019 , piece_id=URV3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.706243235 (59h44m16.57732628s) [noise=true , address=185.167.204.205:4035 , piece_id=6NZETOEX253U2AGIILFBIK3JGBL2S7PJV6WE77ITLVEROV6SZVWA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.710948004 (59h44m16.582031049s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.712808888 (59h44m16.583891933s) [noise=true , address=185.167.204.206:4065 , piece_id=EPW6FRXQZKBOVMTROQYK5O33BBZHIW7BU5HMHKPMOWZNICQJWWMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.193700893 (59h44m17.064783928s) [noise=true , address=81.171.14.70:4031 , piece_id=KPTJZ4FYTXEYVVSLRQD42OL7NGOTXQF5RVPA3PTC4L7W5AZWQNMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.205494490 (59h44m17.076577535s) [noise=true , address=81.171.14.70:4026 , piece_id=UQBSQNBMHZY7733ZI6VNDEU5WXZ2GUBRWPUL3W5LRLWEVLXFN7GQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.213382981 (59h44m17.084466026s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.223902893 (59h44m17.094985938s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.272328052 (59h44m17.143411088s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.314279466 (59h44m17.185362501s) [noise=true , address=81.171.14.70:4038 , piece_id=QWZFMCB3BKGSROA6WTKBMTIKIKZ744UZ7222NGF2ZRMXMJH55K2Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.339184389 (59h44m17.210267434s) [noise=true , address=81.171.14.70:4029 , piece_id=N2MJLXIIMGZZG3TCJ5LJUGH4J5H5OXA6GOGLSBTN6BEMDXOWGTDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.352328829 (59h44m17.223411864s) [noise=true , address=81.171.14.70:4028 , piece_id=K4GHMGJRVY5233RXSDQSHW4OQSXK6A534J36KNIJZNRBLE2YMNOQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.363261126 (59h44m17.234344171s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.770690916 (59h44m17.641773951s) [noise=true , address=81.171.14.70:4027 , piece_id=SRP3XCTIO67QDY2APAFNUGDVJYP4NTNH2ALVZO3CKQWJRGONSDHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.980125476 (59h44m17.851208521s) [noise=true , address=89.149.201.147:4005 , piece_id=DLFODTUO7RTGVF5CKBB5NFBCRLEVBR57I2W746O6JQX4ZJ6Q4RUA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040367989 (59h44m17.911451024s) [noise=true , address=89.149.201.147:4007 , piece_id=33HGXVE4ZG2EH6ROJSXS2WRSZSM3XFH7JXLTHT6QOIADMWWMB4SA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040388418 (59h44m17.911471453s) [seg_pos={0 0}, upload=52461]: Segment reached optimal threshold of 26 pieces. Dec 8 03:13:18.040618335 (59h44m17.91170137s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040631010 (59h44m17.911714055s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040763592 (59h44m17.911846647s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040782438 (59h44m17.911865483s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040832964 (59h44m17.911916009s) [seg_pos={0 0}, upload=52461]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 03:13:18.040964104 (59h44m17.912047149s) [upload=52461]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:13:18.040993821 (59h44m17.912076866s) [upload=52461]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:25:31.019642569 (59h56m30.890725614s) [upload=52461]: Done uploading segments... Dec 8 03:25:31.019743331 (59h56m30.890826366s) [upload=52461]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.1099144s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 703.230806ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6efc88c1a2a7b7f2 FAIL (2025-12-08 03:49:39 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 703.255001ms core satellite us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.302123684s trace: https://jaeger.dev.storj.io/trace/456cfc131726c52f FAIL (2025-12-08 03:49:35 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.302160422s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.840399107s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:49:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.840425558s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.487997075s remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6efc88c1a2a7b7f2 FAIL (2025-12-08 03:49:28 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 1m0.488021572s core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [....X...X.X...........X..........X......X....X...........X..] duration: 3.516922371s trace: https://jaeger.dev.storj.io/trace/59dc391c761605b5 FAIL (2025-12-08 03:49:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.516949283s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.186029735s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:49:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.186044383s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 4.995394291s trace: https://jaeger.dev.storj.io/trace/456cfc131726c52f FAIL (2025-12-08 03:49:14 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.995421333s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.78705938s trace: https://jaeger.dev.storj.io/trace/5ce16d59ac7ca3 FAIL (2025-12-08 03:49:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== V3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478633206 (59h44m15.349716251s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478935492 (59h44m15.350018537s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479212750 (59h44m15.350295785s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479658087 (59h44m15.350741132s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.480729167 (59h44m15.351812212s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481358675 (59h44m15.35244171s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481870881 (59h44m15.352953916s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482011018 (59h44m15.353094063s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482599078 (59h44m15.353682113s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.483043735 (59h44m15.35412679s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:16.296121260 (59h44m16.167204295s) [upload=52461]: Next returned nil segment Dec 8 03:13:16.296136789 (59h44m16.167219824s) [upload=52461]: Waiting for error group managing segments... Dec 8 03:13:16.688782787 (59h44m16.559865822s) [noise=true , address=185.167.204.206:4059 , piece_id=G4AB6LYXSDKJOYKXNAILU34CEGNPXWYJQICBGHLDTWNH3TMHA7JA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.689199911 (59h44m16.560282946s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690209955 (59h44m16.561293s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690460382 (59h44m16.561543427s) [noise=true , address=185.167.204.205:4010 , piece_id=5TOGC4H52AGJS6ZZJJM56DZXZQJ2FBK2YZO6634ACFVLFGOOHC5A, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.696363612 (59h44m16.567446657s) [noise=true , address=185.167.204.206:4051 , piece_id=SPX6V5WZAWUFRZ23KZKQPPWS3VX37O6EOOQTCDRXXNMIGQGSERMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697250752 (59h44m16.568333787s) [noise=true , address=185.167.204.205:4011 , piece_id=ZM5KFXNZTZTU6WS33BQ4HHO2G4EHQGNCZKVNSKXPXEXAM4RW6BQA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697962948 (59h44m16.569045983s) [noise=true , address=185.167.204.206:4072 , piece_id=F5EN43FGL2LXIZU4KHKDGCXM3O7YJM37AT2XBHRLSEOWDJMP5GCA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.700404106 (59h44m16.571487151s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701120280 (59h44m16.572203315s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701632446 (59h44m16.572715481s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.702967407 (59h44m16.574050453s) [noise=true , address=185.167.204.205:4019 , piece_id=URV3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.706243235 (59h44m16.57732628s) [noise=true , address=185.167.204.205:4035 , piece_id=6NZETOEX253U2AGIILFBIK3JGBL2S7PJV6WE77ITLVEROV6SZVWA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.710948004 (59h44m16.582031049s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.712808888 (59h44m16.583891933s) [noise=true , address=185.167.204.206:4065 , piece_id=EPW6FRXQZKBOVMTROQYK5O33BBZHIW7BU5HMHKPMOWZNICQJWWMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.193700893 (59h44m17.064783928s) [noise=true , address=81.171.14.70:4031 , piece_id=KPTJZ4FYTXEYVVSLRQD42OL7NGOTXQF5RVPA3PTC4L7W5AZWQNMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.205494490 (59h44m17.076577535s) [noise=true , address=81.171.14.70:4026 , piece_id=UQBSQNBMHZY7733ZI6VNDEU5WXZ2GUBRWPUL3W5LRLWEVLXFN7GQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.213382981 (59h44m17.084466026s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.223902893 (59h44m17.094985938s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.272328052 (59h44m17.143411088s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.314279466 (59h44m17.185362501s) [noise=true , address=81.171.14.70:4038 , piece_id=QWZFMCB3BKGSROA6WTKBMTIKIKZ744UZ7222NGF2ZRMXMJH55K2Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.339184389 (59h44m17.210267434s) [noise=true , address=81.171.14.70:4029 , piece_id=N2MJLXIIMGZZG3TCJ5LJUGH4J5H5OXA6GOGLSBTN6BEMDXOWGTDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.352328829 (59h44m17.223411864s) [noise=true , address=81.171.14.70:4028 , piece_id=K4GHMGJRVY5233RXSDQSHW4OQSXK6A534J36KNIJZNRBLE2YMNOQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.363261126 (59h44m17.234344171s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.770690916 (59h44m17.641773951s) [noise=true , address=81.171.14.70:4027 , piece_id=SRP3XCTIO67QDY2APAFNUGDVJYP4NTNH2ALVZO3CKQWJRGONSDHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.980125476 (59h44m17.851208521s) [noise=true , address=89.149.201.147:4005 , piece_id=DLFODTUO7RTGVF5CKBB5NFBCRLEVBR57I2W746O6JQX4ZJ6Q4RUA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040367989 (59h44m17.911451024s) [noise=true , address=89.149.201.147:4007 , piece_id=33HGXVE4ZG2EH6ROJSXS2WRSZSM3XFH7JXLTHT6QOIADMWWMB4SA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040388418 (59h44m17.911471453s) [seg_pos={0 0}, upload=52461]: Segment reached optimal threshold of 26 pieces. Dec 8 03:13:18.040618335 (59h44m17.91170137s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040631010 (59h44m17.911714055s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040763592 (59h44m17.911846647s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040782438 (59h44m17.911865483s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040832964 (59h44m17.911916009s) [seg_pos={0 0}, upload=52461]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 03:13:18.040964104 (59h44m17.912047149s) [upload=52461]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:13:18.040993821 (59h44m17.912076866s) [upload=52461]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:25:31.019642569 (59h56m30.890725614s) [upload=52461]: Done uploading segments... Dec 8 03:25:31.019743331 (59h56m30.890826366s) [upload=52461]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.787094799s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X...............X........................] duration: 5.704354086s trace: https://jaeger.dev.storj.io/trace/15c53885b53c2c0b FAIL (2025-12-08 03:49:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.704390886s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..X.........X...............X...................X...........] duration: 5.682496803s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:49:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.682523674s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X......X............................] duration: 2.01307005s trace: https://jaeger.dev.storj.io/trace/67981d3422e72759 FAIL (2025-12-08 03:48:49 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.013098984s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.030218411s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:48:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.030253588s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.....X...X.X...........X..........X......X....X...........X.] duration: 3.468916393s trace: https://jaeger.dev.storj.io/trace/59dc391c761605b5 FAIL (2025-12-08 03:48:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.469012336s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.826252345s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:48:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.826284747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.94249362s trace: https://jaeger.dev.storj.io/trace/456cfc131726c52f FAIL (2025-12-08 03:48:19 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.942522796s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.187800067s trace: https://jaeger.dev.storj.io/trace/5ce16d59ac7ca3 FAIL (2025-12-08 03:48:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== V3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478633206 (59h44m15.349716251s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478935492 (59h44m15.350018537s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479212750 (59h44m15.350295785s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479658087 (59h44m15.350741132s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.480729167 (59h44m15.351812212s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481358675 (59h44m15.35244171s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481870881 (59h44m15.352953916s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482011018 (59h44m15.353094063s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482599078 (59h44m15.353682113s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.483043735 (59h44m15.35412679s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:16.296121260 (59h44m16.167204295s) [upload=52461]: Next returned nil segment Dec 8 03:13:16.296136789 (59h44m16.167219824s) [upload=52461]: Waiting for error group managing segments... Dec 8 03:13:16.688782787 (59h44m16.559865822s) [noise=true , address=185.167.204.206:4059 , piece_id=G4AB6LYXSDKJOYKXNAILU34CEGNPXWYJQICBGHLDTWNH3TMHA7JA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.689199911 (59h44m16.560282946s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690209955 (59h44m16.561293s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690460382 (59h44m16.561543427s) [noise=true , address=185.167.204.205:4010 , piece_id=5TOGC4H52AGJS6ZZJJM56DZXZQJ2FBK2YZO6634ACFVLFGOOHC5A, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.696363612 (59h44m16.567446657s) [noise=true , address=185.167.204.206:4051 , piece_id=SPX6V5WZAWUFRZ23KZKQPPWS3VX37O6EOOQTCDRXXNMIGQGSERMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697250752 (59h44m16.568333787s) [noise=true , address=185.167.204.205:4011 , piece_id=ZM5KFXNZTZTU6WS33BQ4HHO2G4EHQGNCZKVNSKXPXEXAM4RW6BQA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697962948 (59h44m16.569045983s) [noise=true , address=185.167.204.206:4072 , piece_id=F5EN43FGL2LXIZU4KHKDGCXM3O7YJM37AT2XBHRLSEOWDJMP5GCA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.700404106 (59h44m16.571487151s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701120280 (59h44m16.572203315s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701632446 (59h44m16.572715481s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.702967407 (59h44m16.574050453s) [noise=true , address=185.167.204.205:4019 , piece_id=URV3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.706243235 (59h44m16.57732628s) [noise=true , address=185.167.204.205:4035 , piece_id=6NZETOEX253U2AGIILFBIK3JGBL2S7PJV6WE77ITLVEROV6SZVWA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.710948004 (59h44m16.582031049s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.712808888 (59h44m16.583891933s) [noise=true , address=185.167.204.206:4065 , piece_id=EPW6FRXQZKBOVMTROQYK5O33BBZHIW7BU5HMHKPMOWZNICQJWWMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.193700893 (59h44m17.064783928s) [noise=true , address=81.171.14.70:4031 , piece_id=KPTJZ4FYTXEYVVSLRQD42OL7NGOTXQF5RVPA3PTC4L7W5AZWQNMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.205494490 (59h44m17.076577535s) [noise=true , address=81.171.14.70:4026 , piece_id=UQBSQNBMHZY7733ZI6VNDEU5WXZ2GUBRWPUL3W5LRLWEVLXFN7GQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.213382981 (59h44m17.084466026s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.223902893 (59h44m17.094985938s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.272328052 (59h44m17.143411088s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.314279466 (59h44m17.185362501s) [noise=true , address=81.171.14.70:4038 , piece_id=QWZFMCB3BKGSROA6WTKBMTIKIKZ744UZ7222NGF2ZRMXMJH55K2Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.339184389 (59h44m17.210267434s) [noise=true , address=81.171.14.70:4029 , piece_id=N2MJLXIIMGZZG3TCJ5LJUGH4J5H5OXA6GOGLSBTN6BEMDXOWGTDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.352328829 (59h44m17.223411864s) [noise=true , address=81.171.14.70:4028 , piece_id=K4GHMGJRVY5233RXSDQSHW4OQSXK6A534J36KNIJZNRBLE2YMNOQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.363261126 (59h44m17.234344171s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.770690916 (59h44m17.641773951s) [noise=true , address=81.171.14.70:4027 , piece_id=SRP3XCTIO67QDY2APAFNUGDVJYP4NTNH2ALVZO3CKQWJRGONSDHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.980125476 (59h44m17.851208521s) [noise=true , address=89.149.201.147:4005 , piece_id=DLFODTUO7RTGVF5CKBB5NFBCRLEVBR57I2W746O6JQX4ZJ6Q4RUA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040367989 (59h44m17.911451024s) [noise=true , address=89.149.201.147:4007 , piece_id=33HGXVE4ZG2EH6ROJSXS2WRSZSM3XFH7JXLTHT6QOIADMWWMB4SA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040388418 (59h44m17.911471453s) [seg_pos={0 0}, upload=52461]: Segment reached optimal threshold of 26 pieces. Dec 8 03:13:18.040618335 (59h44m17.91170137s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040631010 (59h44m17.911714055s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040763592 (59h44m17.911846647s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040782438 (59h44m17.911865483s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040832964 (59h44m17.911916009s) [seg_pos={0 0}, upload=52461]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 03:13:18.040964104 (59h44m17.912047149s) [upload=52461]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:13:18.040993821 (59h44m17.912076866s) [upload=52461]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:25:31.019642569 (59h56m30.890725614s) [upload=52461]: Done uploading segments... Dec 8 03:25:31.019743331 (59h56m30.890826366s) [upload=52461]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.187816668s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X...............X.......................] duration: 5.779566635s trace: https://jaeger.dev.storj.io/trace/15c53885b53c2c0b FAIL (2025-12-08 03:48:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.779597175s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...X.........X...............X...................X..........] duration: 5.736348191s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:48:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.736374831s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.695471273s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:47:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.695492925s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [X.....X...X.X...........X..........X......X....X...........X] duration: 5.000978435s trace: https://jaeger.dev.storj.io/trace/59dc391c761605b5 FAIL (2025-12-08 03:47:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001001339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X......X...........................] duration: 1.988208424s trace: https://jaeger.dev.storj.io/trace/67981d3422e72759 FAIL (2025-12-08 03:47:45 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.988239453s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.76272188s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:47:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.762757096s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.178547668s trace: https://jaeger.dev.storj.io/trace/456cfc131726c52f FAIL (2025-12-08 03:47:34 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.178575392s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X...............X......................] duration: 5.7803357s trace: https://jaeger.dev.storj.io/trace/15c53885b53c2c0b FAIL (2025-12-08 03:47:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.780364894s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.221804996s trace: https://jaeger.dev.storj.io/trace/5ce16d59ac7ca3 FAIL (2025-12-08 03:47:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== V3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478633206 (59h44m15.349716251s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.478935492 (59h44m15.350018537s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479212750 (59h44m15.350295785s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.479658087 (59h44m15.350741132s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.480729167 (59h44m15.351812212s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481358675 (59h44m15.35244171s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.481870881 (59h44m15.352953916s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482011018 (59h44m15.353094063s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.482599078 (59h44m15.353682113s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:15.483043735 (59h44m15.35412679s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Uploading piece... Dec 8 03:13:16.296121260 (59h44m16.167204295s) [upload=52461]: Next returned nil segment Dec 8 03:13:16.296136789 (59h44m16.167219824s) [upload=52461]: Waiting for error group managing segments... Dec 8 03:13:16.688782787 (59h44m16.559865822s) [noise=true , address=185.167.204.206:4059 , piece_id=G4AB6LYXSDKJOYKXNAILU34CEGNPXWYJQICBGHLDTWNH3TMHA7JA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.689199911 (59h44m16.560282946s) [noise=true , address=185.167.204.206:4043 , piece_id=PEASXJDAI2ORI5ZOH5GMCNMFNXKL2SKKZP3WBQ72VOGB7BJZKWHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690209955 (59h44m16.561293s) [noise=true , address=185.167.204.206:4055 , piece_id=RN3ETZZORZJ5FLRZWXGDTBOJP2ACTVIXO7QSVRH4EY7H4HQNZIKA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.690460382 (59h44m16.561543427s) [noise=true , address=185.167.204.205:4010 , piece_id=5TOGC4H52AGJS6ZZJJM56DZXZQJ2FBK2YZO6634ACFVLFGOOHC5A, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.696363612 (59h44m16.567446657s) [noise=true , address=185.167.204.206:4051 , piece_id=SPX6V5WZAWUFRZ23KZKQPPWS3VX37O6EOOQTCDRXXNMIGQGSERMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697250752 (59h44m16.568333787s) [noise=true , address=185.167.204.205:4011 , piece_id=ZM5KFXNZTZTU6WS33BQ4HHO2G4EHQGNCZKVNSKXPXEXAM4RW6BQA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.697962948 (59h44m16.569045983s) [noise=true , address=185.167.204.206:4072 , piece_id=F5EN43FGL2LXIZU4KHKDGCXM3O7YJM37AT2XBHRLSEOWDJMP5GCA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.700404106 (59h44m16.571487151s) [noise=true , address=185.167.204.206:4050 , piece_id=ZA5QCT6A7AZJL25YXEPHFFBMLXGMIV37WVYZKN6IBGIUYO4UKCDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701120280 (59h44m16.572203315s) [noise=true , address=185.167.204.206:4058 , piece_id=ODSLWPR4JMPSNQSUL3OHUAUKBDCUTUX2UGH2WZDPE7PBOW3NBA3Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.701632446 (59h44m16.572715481s) [noise=true , address=185.167.204.206:4040 , piece_id=ORW7CUV65QLBVVH2SNJRBLINACXMI4C6HXYSMWFUZ7OUAZX5YDPA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.702967407 (59h44m16.574050453s) [noise=true , address=185.167.204.205:4019 , piece_id=URV3YKXL43AUX3HQ4QJ62BOB7R2A55YYYREHC645WOGLRCHEROOA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.706243235 (59h44m16.57732628s) [noise=true , address=185.167.204.205:4035 , piece_id=6NZETOEX253U2AGIILFBIK3JGBL2S7PJV6WE77ITLVEROV6SZVWA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.710948004 (59h44m16.582031049s) [noise=true , address=185.167.204.206:4061 , piece_id=MKP6KY35F7BXGDRUFVWEI6Y7OJXYSIX45452625SPQVGRT4X5GHQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:16.712808888 (59h44m16.583891933s) [noise=true , address=185.167.204.206:4065 , piece_id=EPW6FRXQZKBOVMTROQYK5O33BBZHIW7BU5HMHKPMOWZNICQJWWMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.193700893 (59h44m17.064783928s) [noise=true , address=81.171.14.70:4031 , piece_id=KPTJZ4FYTXEYVVSLRQD42OL7NGOTXQF5RVPA3PTC4L7W5AZWQNMQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.205494490 (59h44m17.076577535s) [noise=true , address=81.171.14.70:4026 , piece_id=UQBSQNBMHZY7733ZI6VNDEU5WXZ2GUBRWPUL3W5LRLWEVLXFN7GQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.213382981 (59h44m17.084466026s) [noise=true , address=81.171.14.70:4032 , piece_id=KJAHFSUBCJ5RSF3R5VSI5M2U7C7AGSE6P76P253SDQT7SBRZ2ZSQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.223902893 (59h44m17.094985938s) [noise=true , address=81.171.14.70:4042 , piece_id=LNABALIBR2GPDGMNG6HKTKLBPSQH3HB5HX67MCMY6LEUV22HX2IA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.272328052 (59h44m17.143411088s) [noise=true , address=81.171.14.70:4025 , piece_id=XGAQ5L2ATTPBNPOA45SCASWNN6JT7L5K5TB5IQSAYLM4QUZQX2MQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.314279466 (59h44m17.185362501s) [noise=true , address=81.171.14.70:4038 , piece_id=QWZFMCB3BKGSROA6WTKBMTIKIKZ744UZ7222NGF2ZRMXMJH55K2Q, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.339184389 (59h44m17.210267434s) [noise=true , address=81.171.14.70:4029 , piece_id=N2MJLXIIMGZZG3TCJ5LJUGH4J5H5OXA6GOGLSBTN6BEMDXOWGTDA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.352328829 (59h44m17.223411864s) [noise=true , address=81.171.14.70:4028 , piece_id=K4GHMGJRVY5233RXSDQSHW4OQSXK6A534J36KNIJZNRBLE2YMNOQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.363261126 (59h44m17.234344171s) [noise=true , address=81.171.14.70:4037 , piece_id=5Q5SC2XQKNY5V26DWVUQXYJJZQE2JXH5RWJWBC4MCGDQYO5ZBIMA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.770690916 (59h44m17.641773951s) [noise=true , address=81.171.14.70:4027 , piece_id=SRP3XCTIO67QDY2APAFNUGDVJYP4NTNH2ALVZO3CKQWJRGONSDHA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:17.980125476 (59h44m17.851208521s) [noise=true , address=89.149.201.147:4005 , piece_id=DLFODTUO7RTGVF5CKBB5NFBCRLEVBR57I2W746O6JQX4ZJ6Q4RUA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040367989 (59h44m17.911451024s) [noise=true , address=89.149.201.147:4007 , piece_id=33HGXVE4ZG2EH6ROJSXS2WRSZSM3XFH7JXLTHT6QOIADMWWMB4SA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: <nil> Dec 8 03:13:18.040388418 (59h44m17.911471453s) [seg_pos={0 0}, upload=52461]: Segment reached optimal threshold of 26 pieces. Dec 8 03:13:18.040618335 (59h44m17.91170137s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040631010 (59h44m17.911714055s) [noise=true , address=89.149.201.147:4002 , piece_id=EXQCZSW5RDE3KJTHWTFJ5GS24FKBKTELCIHD7NPUAPAAF2EWZ6CA, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040763592 (59h44m17.911846647s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Dec 8 03:13:18.040782438 (59h44m17.911865483s) [noise=true , address=89.149.201.147:4021 , piece_id=IGHIV22JPHXISR23YX4PVYDMMFKQM266YOHZ3F37A3PUCU4LRCEQ, seg_pos={0 0}, upload=52461]: Long tail cancellation due to optimal threshold Dec 8 03:13:18.040832964 (59h44m17.911916009s) [seg_pos={0 0}, upload=52461]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 03:13:18.040964104 (59h44m17.912047149s) [upload=52461]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:13:18.040993821 (59h44m17.912076866s) [upload=52461]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:25:31.019642569 (59h56m30.890725614s) [upload=52461]: Done uploading segments... Dec 8 03:25:31.019743331 (59h56m30.890826366s) [upload=52461]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.221851124s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....X.........X...............X...................X.........] duration: 5.75328943s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:47:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.753312783s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X...............X.....................] duration: 5.614216428s trace: https://jaeger.dev.storj.io/trace/15c53885b53c2c0b FAIL (2025-12-08 03:47:00 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.614250402s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.X.....X...X.X...........X..........X......X....X...........] duration: 3.405218593s trace: https://jaeger.dev.storj.io/trace/512a8bd01d6f7b33 FAIL (2025-12-08 03:46:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.405247427s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X......X..........................] duration: 2.051670326s trace: https://jaeger.dev.storj.io/trace/67981d3422e72759 FAIL (2025-12-08 03:46:57 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.051702676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.672899553s trace: https://jaeger.dev.storj.io/trace/181aeb075a359abb FAIL (2025-12-08 03:46:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454589606 (59h45m50.325672641s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.454901751 (59h45m50.325984786s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455220038 (59h45m50.326303083s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455621522 (59h45m50.326704557s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.455956660 (59h45m50.327039695s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456289073 (59h45m50.327372118s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456612098 (59h45m50.327695144s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.456946765 (59h45m50.32802981s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.458680467 (59h45m50.329763512s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.460060705 (59h45m50.33114374s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.461350561 (59h45m50.332433606s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.462929117 (59h45m50.334012162s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.464669651 (59h45m50.335752696s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466275188 (59h45m50.337358223s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.466899267 (59h45m50.337982312s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:50.468267071 (59h45m50.339350106s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Uploading piece... Dec 8 03:14:51.824746486 (59h45m51.695829522s) [upload=52483]: Next returned nil segment Dec 8 03:14:51.824761806 (59h45m51.695844881s) [upload=52483]: Waiting for error group managing segments... Dec 8 03:14:52.168683776 (59h45m52.039766811s) [noise=true , address=64.49.13.139:4479 , piece_id=NDIMXSB7HI2275EIRSZXF7RMLAK7Z4CHIY3KV4FQOW3W3ODNFHCQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.219276294 (59h45m52.090359339s) [noise=true , address=64.49.13.137:4503 , piece_id=XRE3OC6EOYKBSVH5QSJEP3OEMABQICG757PBJF4BO6Y2NZXKLUCA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.232504452 (59h45m52.103587498s) [noise=true , address=64.49.13.138:4484 , piece_id=B662IVDCZXDQD4CRWJWIO2RKJ4FSWOJPXPSJMJP57S7HJZPMWNFQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.359951265 (59h45m52.23103431s) [noise=true , address=77.90.31.19:4845 , piece_id=Y6O57T42S6MJ6OIVVC6RRM7HHUBDAL2IZIIVXSX2ATWEJOLCDJ6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.430948747 (59h45m52.302031782s) [noise=true , address=77.90.31.13:4652 , piece_id=NJFD2U6JYML2P6FVDQHWGCKETKK2DMA7H67RKVMHOTP7FAWJOG6Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442354135 (59h45m52.31343718s) [noise=true , address=77.90.31.12:4609 , piece_id=5JSQXATSAR7IUISI5H2BVLIAWKA7J3NQDYGD4WU5OQAJ3DYTQKZA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.442770528 (59h45m52.313853573s) [noise=true , address=45.91.116.11:4147 , piece_id=JSBISTRA2PFSSBP77R4MYQLMGLSXK37AK5SZWMJAAJXPEPFSZJXA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444206031 (59h45m52.315289076s) [noise=true , address=145.79.227.48:4325 , piece_id=L73GCV6XNO7U4RYUJQ7LDO77ZXRII53HEDSJ3Q5RTDC25NZEJJMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.444711955 (59h45m52.31579499s) [noise=true , address=145.79.227.45:4237 , piece_id=UGE4PKKPY3ZPQNEVY737USGNXPFU2D6UCGR4E75UMMG5H272HJPQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.454853546 (59h45m52.325936581s) [noise=true , address=145.79.227.40:4068 , piece_id=NPBXOCU6KDOOBAA3TFZOUQMX6YX4C2EEMEJJF6U4FSYY5ER7XNJQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.459856934 (59h45m52.330939979s) [noise=true , address=45.91.116.5:4331 , piece_id=TRK7QDEYWWDD4JYB3KLIM63YGXPJCXDCMZBN5XGQIUQSJYTRMO6A, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.469367493 (59h45m52.340450528s) [noise=true , address=45.91.116.4:4969 , piece_id=ERZVYCMZ354H764OBTGAJJLFZIXMT6VPVICU3SRJXFQXHJSTCOPA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.479951517 (59h45m52.351034562s) [noise=true , address=45.91.116.13:4169 , piece_id=LKY7DXT35POC65VJ266C7BHUH7FIDS4Y3EFSQDLP5X6THU4HHHJA, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.515994289 (59h45m52.387077324s) [noise=true , address=45.91.116.6:4998 , piece_id=3PM72VJIDSFQGE5AK3ISSI2ZURYTMBYDHH6DJN37WZZCG3RCWC5Q, seg_pos={0 0}, upload=52483]: Done uploading piece. err: <nil> Dec 8 03:14:52.516015168 (59h45m52.387098203s) [seg_pos={0 0}, upload=52483]: Segment reached optimal threshold of 14 pieces. Dec 8 03:14:52.516331542 (59h45m52.387414577s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BaVAZApAbM17VQYWfZFhWEvsKSrKWHwWiZmG1eaZgZd5iYEKR): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516340358 (59h45m52.387423393s) [noise=true , address=145.79.227.32:4844 , piece_id=ZRXV7BROKDF33HMEI2TPMIY4AVE4IDZH2BKE6MVYDMEF4BM4ZXQQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516550478 (59h45m52.387633513s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516557191 (59h45m52.387640226s) [noise=true , address=77.90.31.16:4759 , piece_id=QXTY3KYIA2AW2NFXAAC4SFODVGDFVW2MKPQJXKGCM5FOF3WX77MQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516655238 (59h45m52.387738273s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pXfQm637dESPkrLBWSNALryxEwooBFdQCh1oyB4emUFes2pvx): protocol: expected piece hash; context canceled; EOF Dec 8 03:14:52.516661650 (59h45m52.387744685s) [noise=true , address=145.79.227.17:4990 , piece_id=D2X4UEGUYN756GRJRTYRPCAM6PQUJT3K6KFYJO4WVD2SWQDGVEUQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516879334 (59h45m52.387962379s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17C7Wx6yE15jDtqqsQwWVj444YkvHBUWkkXjoCxfXPiErwyGKQ): protocol: read tcp 188.166.225.170:54622->145.79.227.12:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 03:14:52.516887290 (59h45m52.387970335s) [noise=true , address=145.79.227.12:4782 , piece_id=YEG3OATW7OFRQKG3PVIG43BYFTLJECFUDN4VIVZMOF5UJD6MLYMQ, seg_pos={0 0}, upload=52483]: Long tail cancellation due to optimal threshold Dec 8 03:14:52.516909502 (59h45m52.387992537s) [seg_pos={0 0}, upload=52483]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 03:14:52.517012949 (59h45m52.388095984s) [upload=52483]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 03:14:52.517018219 (59h45m52.388101264s) [upload=52483]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 03:26:03.308190575 (59h57m3.17927362s) [upload=52483]: Done uploading segments... Dec 8 03:26:03.308267711 (59h57m3.179350766s) [upload=52483]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.672915353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.825127083s trace: https://jaeger.dev.storj.io/trace/456cfc131726c52f FAIL (2025-12-08 03:46:50 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.825160575s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....X.........X...............X...................X........] duration: 5.805685661s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:46:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.8057165s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..X.....X...X.X...........X..........X......X....X..........] duration: 4.282799826s trace: https://jaeger.dev.storj.io/trace/512a8bd01d6f7b33 FAIL (2025-12-08 03:46:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.282820314s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......X.........X...............X...................X.......] duration: 6.007772428s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:46:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.007803157s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......X.........X...............X...................X......] duration: 5.652783708s trace: https://jaeger.dev.storj.io/trace/5ab4f67e7dfb3fb FAIL (2025-12-08 03:46:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.652813104s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)