Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 23 out of 60 is failed (threshold=6) [.XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX....XX] duration: 5.001270531s trace: https://jaeger.dev.storj.io/trace/1b1d210f25701f5b FAIL (2025-12-05 15:26:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001295609s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 24.025131344s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:25:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.025156273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX....X] duration: 5.000793736s trace: https://jaeger.dev.storj.io/trace/6c971ba92e447fd8 FAIL (2025-12-05 15:25:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000814094s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X....X..] duration: 7.096763255s trace: https://jaeger.dev.storj.io/trace/5c31c4be77682f4 FAIL (2025-12-05 15:25:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 7.097275089s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 9.229282234s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:25:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 9.229318003s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 8.664648764s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:24:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 8.664674554s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX....] duration: 4.341010655s trace: https://jaeger.dev.storj.io/trace/4c47e9841db2f345 FAIL (2025-12-05 15:24:50 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.341066401s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.902137031s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:24:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.90216327s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration: 18.477788811s trace: https://jaeger.dev.storj.io/trace/5c31c4be77682f4 FAIL (2025-12-05 15:24:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 18.477818127s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 9.023852337s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:24:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 9.023877645s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 10.465426817s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:23:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.465454198s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 22 out of 60 is failed (threshold=6) [.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX...] duration: 3.970837415s trace: https://jaeger.dev.storj.io/trace/4c47e9841db2f345 FAIL (2025-12-05 15:23:40 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.970863065s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X....X] duration: 10.571636011s trace: https://jaeger.dev.storj.io/trace/5c31c4be77682f4 FAIL (2025-12-05 15:23:31 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 10.571662662s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 9.23735348s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:23:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 9.237383959s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 23 out of 60 is failed (threshold=6) [X.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX..] duration: 5.001248548s trace: https://jaeger.dev.storj.io/trace/4c47e9841db2f345 FAIL (2025-12-05 15:23:06 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001276001s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 13.674967128s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:23:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.674996713s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 9.8953972s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:22:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 9.895424752s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 8.763062639s trace: https://jaeger.dev.storj.io/trace/2b4117dfadcbd59d FAIL (2025-12-05 15:22:37 UTC) orbiter warning: context deadline exceeded Duration: 8.763088619s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 9.499936809s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:22:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.499973339s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 19.76069024s trace: https://jaeger.dev.storj.io/trace/2b4117dfadcbd59d FAIL (2025-12-05 15:22:18 UTC) orbiter warning: context deadline exceeded Duration: 19.760703695s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 8.896185925s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:22:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 8.896211493s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [XX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX.] duration: 3.424594424s trace: https://jaeger.dev.storj.io/trace/4c47e9841db2f345 FAIL (2025-12-05 15:21:59 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.424619151s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 26.522792774s trace: https://jaeger.dev.storj.io/trace/2b4117dfadcbd59d FAIL (2025-12-05 15:21:58 UTC) orbiter warning: context deadline exceeded Duration: 26.522818121s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.3517563s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:21:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.351775728s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 8.450886816s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:21:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 8.450916942s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 14.882589805s trace: https://jaeger.dev.storj.io/trace/2b4117dfadcbd59d FAIL (2025-12-05 15:21:07 UTC) orbiter warning: context deadline exceeded Duration: 14.882620473s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [XXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XXX] duration: 5.001676786s trace: https://jaeger.dev.storj.io/trace/4c47e9841db2f345 FAIL (2025-12-05 15:20:58 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001698748s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 21.193162012s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:20:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 21.193196128s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 14.046791327s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:20:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.046814881s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001264603s trace: https://jaeger.dev.storj.io/trace/2b4117dfadcbd59d FAIL (2025-12-05 15:20:17 UTC) orbiter warning: context deadline exceeded Duration: 30.001307244s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 58.937945194s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:20:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 58.937972566s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [XXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....XX] duration: 5.000884477s trace: https://jaeger.dev.storj.io/trace/625f02a6965bcf06 FAIL (2025-12-05 15:19:53 UTC) orbiter warning: context deadline exceeded Duration: 5.000898835s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 17.969359321s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:19:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 17.969387393s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 7.666873176s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:18:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.666897543s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....X] duration: 5.000920747s trace: https://jaeger.dev.storj.io/trace/364e6370de65ebe6 FAIL (2025-12-05 15:18:55 UTC) orbiter warning: context deadline exceeded Duration: 5.000941026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.003082116s trace: https://jaeger.dev.storj.io/trace/5f5ddbd5d126a8c3 FAIL (2025-12-05 15:18:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq, Piece ID: JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A) context deadline exceeded; piecestore: (Node ID: 16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, Piece ID: R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA) context deadline exceeded; piecestore: (Node ID: 1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, Piece ID: 7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ) context deadline exceeded ======LOGS====== Dec 5 15:18:13.604544904 (46h22m50.075755643s) [upload=40767]: Starting upload Dec 5 15:18:13.604676043 (46h22m50.075886783s) [upload=40767]: Done starting upload Dec 5 15:18:13.604782897 (46h22m50.075993636s) [upload=40767]: Uploading segments... Dec 5 15:18:13.605017703 (46h22m50.076228442s) [upload=40767]: Got next segment. Inline: false Dec 5 15:18:13.605071326 (46h22m50.076282065s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 5 15:18:13.605077568 (46h22m50.076288297s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 5 15:18:13.982145827 (46h22m50.453356596s) [seg_pos={0 0}, upload=40767]: Begin upload segment... Dec 5 15:18:13.982241459 (46h22m50.453452188s) [seg_pos={0 0}, upload=40767]: Done begin upload segment. Dec 5 15:18:13.982305261 (46h22m50.45351599s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.982325529 (46h22m50.453536258s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.983716137 (46h22m50.454926876s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984393357 (46h22m50.455604096s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.984496704 (46h22m50.455707443s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985163543 (46h22m50.456374282s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985226954 (46h22m50.456437693s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985692260 (46h22m50.456902999s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.985979356 (46h22m50.457190095s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.986752999 (46h22m50.457963738s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987586206 (46h22m50.458796945s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.987679414 (46h22m50.458890153s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.989391654 (46h22m50.460602393s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.991872307 (46h22m50.463083046s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:13.993669820 (46h22m50.464880569s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:14.016626324 (46h22m50.487837063s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Uploading piece... Dec 5 15:18:15.760943112 (46h22m52.232153841s) [upload=40767]: Next returned nil segment Dec 5 15:18:15.760958311 (46h22m52.23216905s) [upload=40767]: Waiting for error group managing segments... Dec 5 15:18:16.106807332 (46h22m52.578018071s) [noise=true , address=77.90.31.11:4554 , piece_id=RVGUBUKVLBV43QNDKDL7BFDV44LB2QJRULGYRVOXDSDBB32DUVHA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.139997983 (46h22m52.611208723s) [noise=true , address=145.79.227.14:4840 , piece_id=RGXAEMI3AXDOEHLLS46F2SCPSITPV7J5KP7WQTTVZ2GAJ34OJAVQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.177173714 (46h22m52.648384453s) [noise=true , address=145.79.227.44:4193 , piece_id=BHNTOCEQ6M6GOVDACWQXT4ELRNBC7WVYBM7F7DDU5ADLZ7RSKHSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.184863837 (46h22m52.656074576s) [noise=true , address=145.79.227.29:4942 , piece_id=54OLWPYTZUXZH6PBKQQGUJFEH5MKR27T6Y5EUP4DNWR7F5WZYUHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.186542022 (46h22m52.657752751s) [noise=true , address=45.91.116.5:4298 , piece_id=5QXJTLYEJRMQAYHCUL5LF6XPC5MT364BQZHB33QMV2G3HRK2EYHQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.205978523 (46h22m52.677189262s) [noise=true , address=145.79.227.32:4835 , piece_id=R6ROGUTBUCJ5ED6M5AZJUSQAKQCNUPMS3FYWSDL5JASJZ7UJ63YA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.234199571 (46h22m52.705410301s) [noise=true , address=145.79.227.27:4689 , piece_id=3C2EINBMPCL43A54J3ACIRNGRH3SZYLKJJKYU2UJVSG5R3QLIAJQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.238415088 (46h22m52.709625827s) [noise=true , address=145.79.227.28:4726 , piece_id=JSN2UQNM7YSE7NII7M5VXB3KYT72J7WWMOK6OKF447XAQGBINUXQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.242491691 (46h22m52.71370243s) [noise=true , address=145.79.227.32:4834 , piece_id=7NFLHANWBMXXNL5UVL4OMSFHUY3XYIFTRVGRQBK4RZRWD26TXJFQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.285119152 (46h22m52.756329881s) [noise=true , address=145.79.227.43:4158 , piece_id=LYDQ34OBNREUDKR7CIF6HWSFSU7PGHF3TLYLP7XSAKFPXIGEMTPQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.319442550 (46h22m52.790653279s) [noise=true , address=145.79.227.9:4365 , piece_id=JQDONT2TDWZYEU6IHSUJI5EOE7XDCUMB3QNWQEAGJ3JWEC53BH2A, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.346189852 (46h22m52.817400591s) [noise=true , address=145.79.227.20:4438 , piece_id=AAVK3MT32CTLIM4MB3LGHLCYTL2BEUQ5JFCLGAKSYE4RXLP7AIZA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403366707 (46h22m52.874577446s) [noise=true , address=145.79.227.10:4403 , piece_id=7VWASAPLCC6CABUDUDFARBHRJRHSUI5EMI2OTIINFMEVZUUIBFSA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: <nil> Dec 5 15:18:16.403445919 (46h22m52.874656658s) [seg_pos={0 0}, upload=40767]: Segment reached optimal threshold of 13 pieces. Dec 5 15:18:16.403755277 (46h22m52.874966006s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403765367 (46h22m52.874976106s) [noise=true , address=145.79.227.17:4970 , piece_id=JVSON3JX4TIN2X4H6ZUUEGUUESIF7BMI2YTSE7YJUA7MBPDEVGAA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.403942434 (46h22m52.875153173s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.403948295 (46h22m52.875159034s) [noise=true , address=77.90.31.16:4735 , piece_id=MVV75YFBPVISNX73RMUKHFERZYIESQIEHMX3XZZGCMSEBFB4GTGQ, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404277792 (46h22m52.875488521s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcGHVSm5UAXPh5vCfyRHCd7Bdb67oXF51AZcnRFUjXqVYzXLR): protocol: expected piece hash; context canceled; EOF Dec 5 15:18:16.404299754 (46h22m52.875510483s) [noise=true , address=145.79.227.38:4978 , piece_id=FU5YGA3AGYK25JLT5VZGPTP3JZI4FDR5QQD4WCS5BNFBBZWUQ5AA, seg_pos={0 0}, upload=40767]: Long tail cancellation due to optimal threshold Dec 5 15:18:16.404316717 (46h22m52.875527446s) [seg_pos={0 0}, upload=40767]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 5 15:18:16.404397901 (46h22m52.87560864s) [upload=40767]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 5 15:18:16.404409002 (46h22m52.875619741s) [upload=40767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 5 15:18:16.728818930 (46h22m53.200029659s) [upload=40767]: Done uploading segments... Dec 5 15:18:16.728837856 (46h22m53.200048595s) [upload=40767]: Upload finished. err: <nil> Duration: 30.003100923s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 9.664478914s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:18:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.664540913s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [.XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.....] duration: 3.413146243s trace: https://jaeger.dev.storj.io/trace/65989525e09bc6ea FAIL (2025-12-05 15:18:06 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.413181911s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 9.576602602s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:17:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.576630866s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [..XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X....] duration: 3.810289561s trace: https://jaeger.dev.storj.io/trace/65989525e09bc6ea FAIL (2025-12-05 15:17:20 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.810335147s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [...XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X...] duration: 3.496226794s trace: https://jaeger.dev.storj.io/trace/65989525e09bc6ea FAIL (2025-12-05 15:16:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.496253255s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.877069321s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:16:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.877098929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 24 out of 60 is failed (threshold=6) [....XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X..] duration: 5.000721167s trace: https://jaeger.dev.storj.io/trace/65989525e09bc6ea FAIL (2025-12-05 15:16:12 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000765492s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.418831996s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:15:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.418864787s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [X....XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X.] duration: 4.04353313s trace: https://jaeger.dev.storj.io/trace/65989525e09bc6ea FAIL (2025-12-05 15:15:18 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.043558879s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [.X....XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..X] duration: 5.00140073s trace: https://jaeger.dev.storj.io/trace/65989525e09bc6ea FAIL (2025-12-05 15:14:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001423282s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 15.739274499s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:14:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.739306219s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 25 out of 60 is failed (threshold=6) [X.X....XXXXX.X..XX.......X...X..X..X...X..XX.XX..XXX.XX..X..] duration: 4.335522014s trace: https://jaeger.dev.storj.io/trace/551ee2c3676bb0d8 FAIL (2025-12-05 15:14:11 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.33555189s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.526513007s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:13:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.526526184s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 17.957793154s trace: https://jaeger.dev.storj.io/trace/7a8f99435637708e FAIL (2025-12-05 15:13:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.957824284s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)