Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 16 out of 60 is failed (threshold=3) [X..XXX....XX......XX..XX....X............XXX..............XX] duration: 30.000629678s FAIL (2025-08-26 05:58:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000653112s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [.XX....X..X..X..X.......XX.....XXXXX.....XX.............X.X.] duration: 20.147168991s FAIL (2025-08-26 05:58:01 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 20.147197333s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.X..XX.....X...................X.........XX.................] duration: 19.565221183s FAIL (2025-08-26 05:57:59 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:19.122249242 (13h42m34.347432309s) [upload=12027]: Starting upload Aug 26 05:46:19.122336606 (13h42m34.347519673s) [upload=12027]: Done starting upload Aug 26 05:46:19.122389295 (13h42m34.347572352s) [upload=12027]: Uploading segments... Aug 26 05:46:19.122426304 (13h42m34.347609352s) [upload=12027]: Got next segment. Inline: true Aug 26 05:46:19.122431003 (13h42m34.34761406s) [upload=12027]: Waiting for error group managing segments... Aug 26 05:46:19.122482891 (13h42m34.347665938s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:19.122487078 (13h42m34.347670135s) [upload=12027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:19.122489442 (13h42m34.3476725s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:30.124196038 (13h42m45.349379106s) [upload=12027]: Done uploading segments... Aug 26 05:46:30.124211648 (13h42m45.349394705s) [upload=12027]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 19.565251261s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 11 out of 60 is failed (threshold=3) [XX.X.X.X.....X..X......................X.....X..X........X..] duration: 21.485069633s FAIL (2025-08-26 05:57:56 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 21.485089359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [X.X.X.XX....X..X...XX.XXX..........X..XX..................XX] duration: 30.000392259s FAIL (2025-08-26 05:57:55 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:57:25.146232621 (13h53m40.371415688s) [upload=12188]: Starting upload Aug 26 05:57:25.146338740 (13h53m40.371521818s) [upload=12188]: Done starting upload Aug 26 05:57:25.146402892 (13h53m40.371585979s) [upload=12188]: Uploading segments... Aug 26 05:57:25.146452114 (13h53m40.371635192s) [upload=12188]: Got next segment. Inline: true Aug 26 05:57:25.146458015 (13h53m40.371641092s) [upload=12188]: Waiting for error group managing segments... Aug 26 05:57:25.146498010 (13h53m40.371681078s) [upload=12188]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:57:25.146502859 (13h53m40.371685927s) [upload=12188]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:57:25.146506517 (13h53m40.371689593s) [upload=12188]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:57:37.270881730 (13h53m52.496064867s) [upload=12188]: Done uploading segments... Aug 26 05:57:37.270909052 (13h53m52.496092129s) [upload=12188]: Upload finished. err: <nil> Duration:* 30.000407518s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [X....XXX......X....X...X......XX.X.XX....XX.XX.........X.XX.] duration: 23.021481098s FAIL (2025-08-26 05:57:55 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3B81C9C1F624, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 23.021515251s edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 3 out of 60 is failed (threshold=6) [....X....X.......................X..........................] duration: 8.728178545s remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD X-Request-Id=ZFwx4y7eAxE Content-Type=text/html; charset=utf-8 Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:38:38 GMT target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:57:47 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 8.728239849s edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XX.XXXXX....X..X...X...X......X..XX.X.......XX..........XXX.] duration: 21.920788565s FAIL (2025-08-26 05:57:43 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:56:49.689887577 (13h53m4.915070634s) [upload=12180]: Starting upload Aug 26 05:56:49.689930739 (13h53m4.915113776s) [upload=12180]: Done starting upload Aug 26 05:56:49.690004758 (13h53m4.915187815s) [upload=12180]: Uploading segments... Aug 26 05:56:49.690105518 (13h53m4.915288565s) [upload=12180]: Got next segment. Inline: false Aug 26 05:56:49.690129613 (13h53m4.91531266s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:49.690133130 (13h53m4.915316177s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:49.809939327 (13h53m5.035122374s) [seg_pos={0 0}, upload=12180]: Begin upload segment... Aug 26 05:56:49.809970326 (13h53m5.035153383s) [seg_pos={0 0}, upload=12180]: Done begin upload segment. Aug 26 05:56:49.809989402 (13h53m5.035172449s) [noise=true , address=45.91.116.6:4975 , piece_id=NGMDRK6UBQBA25W7QUYA32DR54EQNRFKOGYHMCU6VCW2523QPGPA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810007536 (13h53m5.035190593s) [noise=true , address=104.36.82.8:4969 , piece_id=I4F3Q7CAMFKBEYGX3GSZ5SSQSYHADMRBFTGC6TWSWN3HDAC3Y2YQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810295448 (13h53m5.035478495s) [noise=true , address=45.91.116.4:4938 , piece_id=U2NTGDYIBJGM4C2PH3VXXDLFMPAVOYH7ZEFL4DGNH27GMECXLRVA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810349980 (13h53m5.035533037s) [noise=true , address=45.91.116.14:4189 , piece_id=6WH5SDDZRHSRR6N623JHSVZFRDIPJ7GCBBTGOB6AMRQ6GH4FDVNQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810510963 (13h53m5.03569401s) [noise=true , address=45.91.116.5:4297 , piece_id=6EUXSCDMS2XPFBX465EGCMJZFZH2O2RZQISTV2XODEPKHTVDOMOQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810658120 (13h53m5.035841177s) [noise=true , address=77.90.31.21:4879 , piece_id=JNTF36FINGPPYDKR7OV2GOLMFFHZUZV2TNK5VVLFVCDT3IUSFGPQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810721399 (13h53m5.035904446s) [noise=true , address=45.91.116.12:4226 , piece_id=SRNYOYHMUBOCNMLDATC4EUPPPBWSINM2U3NSFNCUJXRP7ZQIBVMA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810949919 (13h53m5.036132976s) [noise=true , address=45.91.116.12:4225 , piece_id=I3G7OLIULCH4YQO62WU7YKBHYMC5VVQXAULCH5UQCOD7U3E2HD3Q, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811009752 (13h53m5.036192809s) [noise=true , address=77.90.31.21:4877 , piece_id=5CI3JSRNRSDIZAWWHFRIIHJWZQV4BYYPCBNCVOYTYS2FTJCQKDZQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811243541 (13h53m5.036426598s) [noise=true , address=45.91.116.14:4190 , piece_id=SDU2YXVNCAXKUZD27FXV5QMQKELZMB6JFXPA63BK6N4JUONFXERA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811252668 (13h53m5.036435725s) [noise=true , address=77.90.31.18:4805 , piece_id=S5WGLC2RQOZPNSAGNQSVSOODCR7KJ46MGMWOQDPMPGT5CDNVRCKQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811463595 (13h53m5.036646652s) [noise=true , address=77.90.31.13:4628 , piece_id=PXADU2MH7TXQU57ZQWMGSLYJ2SHP6YPH6UBOJFK4KPHXNBRLY6DQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811518559 (13h53m5.036701606s) [noise=true , address=77.90.31.16:4735 , piece_id=OSQLWS6BGSBR2RGF2OAISHGDG3VPGC3UGFVL2WLPZMLQYO5AINIQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811676625 (13h53m5.036859673s) [noise=true , address=77.90.31.11:4554 , piece_id=GENPAW6S7BKJ5AZJCXPAJTVF4SOTL5UA4T4E3GPP4C6BH36EE5CA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811791101 (13h53m5.036974148s) [noise=true , address=45.91.116.10:4081 , piece_id=TBKB4HNCUD7C44E5VJRPIDYXX4RBDYCYZ3QAAF4KN7TUSFOZB74A, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811928379 (13h53m5.037111427s) [noise=true , address=45.91.116.8:4045 , piece_id=BOBGOSJ5RZP7BLVDKQQ6E255QLUQTJLAOK3AFK2PSEFR225TJ2XQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.882896864 (13h53m5.108079921s) [upload=12180]: Next returned nil segment Aug 26 05:56:49.882907514 (13h53m5.108090571s) [upload=12180]: Waiting for error group managing segments... Aug 26 05:56:49.893120667 (13h53m5.118303714s) [noise=true , address=45.91.116.10:4081 , piece_id=TBKB4HNCUD7C44E5VJRPIDYXX4RBDYCYZ3QAAF4KN7TUSFOZB74A, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.893997816 (13h53m5.119180874s) [noise=true , address=45.91.116.14:4190 , piece_id=SDU2YXVNCAXKUZD27FXV5QMQKELZMB6JFXPA63BK6N4JUONFXERA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894143190 (13h53m5.119326237s) [noise=true , address=45.91.116.14:4189 , piece_id=6WH5SDDZRHSRR6N623JHSVZFRDIPJ7GCBBTGOB6AMRQ6GH4FDVNQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894396877 (13h53m5.119579934s) [noise=true , address=45.91.116.8:4045 , piece_id=BOBGOSJ5RZP7BLVDKQQ6E255QLUQTJLAOK3AFK2PSEFR225TJ2XQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894549414 (13h53m5.119732461s) [noise=true , address=77.90.31.21:4877 , piece_id=5CI3JSRNRSDIZAWWHFRIIHJWZQV4BYYPCBNCVOYTYS2FTJCQKDZQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894736436 (13h53m5.119919493s) [noise=true , address=45.91.116.12:4225 , piece_id=I3G7OLIULCH4YQO62WU7YKBHYMC5VVQXAULCH5UQCOD7U3E2HD3Q, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894993890 (13h53m5.120176937s) [noise=true , address=77.90.31.16:4735 , piece_id=OSQLWS6BGSBR2RGF2OAISHGDG3VPGC3UGFVL2WLPZMLQYO5AINIQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.895078880 (13h53m5.120261937s) [noise=true , address=45.91.116.5:4297 , piece_id=6EUXSCDMS2XPFBX465EGCMJZFZH2O2RZQISTV2XODEPKHTVDOMOQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.895359267 (13h53m5.120542314s) [noise=true , address=45.91.116.6:4975 , piece_id=NGMDRK6UBQBA25W7QUYA32DR54EQNRFKOGYHMCU6VCW2523QPGPA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.895424841 (13h53m5.120607898s) [noise=true , address=77.90.31.21:4879 , piece_id=JNTF36FINGPPYDKR7OV2GOLMFFHZUZV2TNK5VVLFVCDT3IUSFGPQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.896055406 (13h53m5.121238454s) [noise=true , address=45.91.116.4:4938 , piece_id=U2NTGDYIBJGM4C2PH3VXXDLFMPAVOYH7ZEFL4DGNH27GMECXLRVA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.897951745 (13h53m5.123134791s) [noise=true , address=77.90.31.13:4628 , piece_id=PXADU2MH7TXQU57ZQWMGSLYJ2SHP6YPH6UBOJFK4KPHXNBRLY6DQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.898959220 (13h53m5.124142267s) [noise=true , address=77.90.31.11:4554 , piece_id=GENPAW6S7BKJ5AZJCXPAJTVF4SOTL5UA4T4E3GPP4C6BH36EE5CA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.898974528 (13h53m5.124157576s) [seg_pos={0 0}, upload=12180]: Segment reached optimal threshold of 13 pieces. Aug 26 05:56:49.899042105 (13h53m5.124225152s) [noise=true , address=104.36.82.8:4969 , piece_id=I4F3Q7CAMFKBEYGX3GSZ5SSQSYHADMRBFTGC6TWSWN3HDAC3Y2YQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:49.899080618 (13h53m5.124263666s) [noise=true , address=77.90.31.18:4805 , piece_id=S5WGLC2RQOZPNSAGNQSVSOODCR7KJ46MGMWOQDPMPGT5CDNVRCKQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:49.899108991 (13h53m5.124292039s) [noise=true , address=45.91.116.12:4226 , piece_id=SRNYOYHMUBOCNMLDATC4EUPPPBWSINM2U3NSFNCUJXRP7ZQIBVMA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:49.899133838 (13h53m5.124316895s) [seg_pos={0 0}, upload=12180]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:56:49.899232344 (13h53m5.124415401s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:49.899236652 (13h53m5.124419709s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:50.107204433 (13h53m5.332387509s) [upload=12180]: Done uploading segments... Aug 26 05:56:50.107285776 (13h53m5.332468852s) [upload=12180]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 21.920809986s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [X.X..XX.....X...................X.........XX................] duration: 45.959881594s FAIL (2025-08-26 05:57:38 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:19.122249242 (13h42m34.347432309s) [upload=12027]: Starting upload Aug 26 05:46:19.122336606 (13h42m34.347519673s) [upload=12027]: Done starting upload Aug 26 05:46:19.122389295 (13h42m34.347572352s) [upload=12027]: Uploading segments... Aug 26 05:46:19.122426304 (13h42m34.347609352s) [upload=12027]: Got next segment. Inline: true Aug 26 05:46:19.122431003 (13h42m34.34761406s) [upload=12027]: Waiting for error group managing segments... Aug 26 05:46:19.122482891 (13h42m34.347665938s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:19.122487078 (13h42m34.347670135s) [upload=12027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:19.122489442 (13h42m34.3476725s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:30.124196038 (13h42m45.349379106s) [upload=12027]: Done uploading segments... Aug 26 05:46:30.124211648 (13h42m45.349394705s) [upload=12027]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 45.959913944s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 11 out of 60 is failed (threshold=3) [.XX.XX....XX.................X...XX........X.............X..] duration: 34.26201004s remote: "tcp:45.38.24.2:443" response headers: X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:56:12 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Content-Length=433 Server=Storj Vary=Origin X-Amz-Request-Id=185F3B796D1B7776 X-Request-Id=JvmnWJy1rYQ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:57:34 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B796D1B7776</RequestId><HostId></HostId></Error>" Duration: 34.262039647s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 11 out of 60 is failed (threshold=3) [.XX.X.X.X.....X..X......................X.....X..X........X.] duration: 24.647416745s FAIL (2025-08-26 05:57:25 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 24.647443335s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 18 out of 60 is failed (threshold=3) [X.XX....X..X..X..X.......XX.....XXXXX.....XX.............X.X] duration: 30.001676007s FAIL (2025-08-26 05:57:18 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001695223s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [X...............X..X.........XX.........X.XX.X..............] duration: 6.43412112s remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:57:16 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 6.434144895s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 25 out of 60 is failed (threshold=3) [XX.XX.......X..XX.XXXXXXX.....XX..X.XXX......XX.X........XX.] duration: 29.102806977s FAIL (2025-08-26 05:57:11 UTC) operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 29.102832014s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [XX.X.X.XX....X..X...XX.XXX..........X..XX..................X] duration: 30.00088521s FAIL (2025-08-26 05:57:10 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:56:40.432645125 (13h52m55.657828182s) [upload=12174]: Starting upload Aug 26 05:56:40.432695881 (13h52m55.657878927s) [upload=12174]: Done starting upload Aug 26 05:56:40.432746887 (13h52m55.657929944s) [upload=12174]: Uploading segments... Aug 26 05:56:40.432781052 (13h52m55.657964109s) [upload=12174]: Got next segment. Inline: true Aug 26 05:56:40.432784899 (13h52m55.657967946s) [upload=12174]: Waiting for error group managing segments... Aug 26 05:56:40.432816287 (13h52m55.657999335s) [upload=12174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:40.432819744 (13h52m55.658002791s) [upload=12174]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:56:40.432822008 (13h52m55.658005065s) [upload=12174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:52.491979360 (13h53m7.717162417s) [upload=12174]: Done uploading segments... Aug 26 05:56:52.491992715 (13h53m7.717175772s) [upload=12174]: Upload finished. err: <nil> Duration:* 30.000906761s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.X..X.............XX...............XX........X..............] duration: 17.297868726s FAIL (2025-08-26 05:57:10 UTC) operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 17.29788702s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 4 out of 60 is failed (threshold=6) [X....X....X.......................X.........................] duration: 5.033716302s remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD X-Request-Id=ZFwx4y7eAxE Content-Type=text/html; charset=utf-8 Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:38:38 GMT target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:56:53 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 5.03374134s edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.XX.XXXXX....X..X...X...X......X..XX.X.......XX..........XXX] duration: 430.992823ms FAIL (2025-08-26 05:56:50 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:56:49.689887577 (13h53m4.915070634s) [upload=12180]: Starting upload Aug 26 05:56:49.689930739 (13h53m4.915113776s) [upload=12180]: Done starting upload Aug 26 05:56:49.690004758 (13h53m4.915187815s) [upload=12180]: Uploading segments... Aug 26 05:56:49.690105518 (13h53m4.915288565s) [upload=12180]: Got next segment. Inline: false Aug 26 05:56:49.690129613 (13h53m4.91531266s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:49.690133130 (13h53m4.915316177s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:49.809939327 (13h53m5.035122374s) [seg_pos={0 0}, upload=12180]: Begin upload segment... Aug 26 05:56:49.809970326 (13h53m5.035153383s) [seg_pos={0 0}, upload=12180]: Done begin upload segment. Aug 26 05:56:49.809989402 (13h53m5.035172449s) [noise=true , address=45.91.116.6:4975 , piece_id=NGMDRK6UBQBA25W7QUYA32DR54EQNRFKOGYHMCU6VCW2523QPGPA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810007536 (13h53m5.035190593s) [noise=true , address=104.36.82.8:4969 , piece_id=I4F3Q7CAMFKBEYGX3GSZ5SSQSYHADMRBFTGC6TWSWN3HDAC3Y2YQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810295448 (13h53m5.035478495s) [noise=true , address=45.91.116.4:4938 , piece_id=U2NTGDYIBJGM4C2PH3VXXDLFMPAVOYH7ZEFL4DGNH27GMECXLRVA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810349980 (13h53m5.035533037s) [noise=true , address=45.91.116.14:4189 , piece_id=6WH5SDDZRHSRR6N623JHSVZFRDIPJ7GCBBTGOB6AMRQ6GH4FDVNQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810510963 (13h53m5.03569401s) [noise=true , address=45.91.116.5:4297 , piece_id=6EUXSCDMS2XPFBX465EGCMJZFZH2O2RZQISTV2XODEPKHTVDOMOQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810658120 (13h53m5.035841177s) [noise=true , address=77.90.31.21:4879 , piece_id=JNTF36FINGPPYDKR7OV2GOLMFFHZUZV2TNK5VVLFVCDT3IUSFGPQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810721399 (13h53m5.035904446s) [noise=true , address=45.91.116.12:4226 , piece_id=SRNYOYHMUBOCNMLDATC4EUPPPBWSINM2U3NSFNCUJXRP7ZQIBVMA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.810949919 (13h53m5.036132976s) [noise=true , address=45.91.116.12:4225 , piece_id=I3G7OLIULCH4YQO62WU7YKBHYMC5VVQXAULCH5UQCOD7U3E2HD3Q, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811009752 (13h53m5.036192809s) [noise=true , address=77.90.31.21:4877 , piece_id=5CI3JSRNRSDIZAWWHFRIIHJWZQV4BYYPCBNCVOYTYS2FTJCQKDZQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811243541 (13h53m5.036426598s) [noise=true , address=45.91.116.14:4190 , piece_id=SDU2YXVNCAXKUZD27FXV5QMQKELZMB6JFXPA63BK6N4JUONFXERA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811252668 (13h53m5.036435725s) [noise=true , address=77.90.31.18:4805 , piece_id=S5WGLC2RQOZPNSAGNQSVSOODCR7KJ46MGMWOQDPMPGT5CDNVRCKQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811463595 (13h53m5.036646652s) [noise=true , address=77.90.31.13:4628 , piece_id=PXADU2MH7TXQU57ZQWMGSLYJ2SHP6YPH6UBOJFK4KPHXNBRLY6DQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811518559 (13h53m5.036701606s) [noise=true , address=77.90.31.16:4735 , piece_id=OSQLWS6BGSBR2RGF2OAISHGDG3VPGC3UGFVL2WLPZMLQYO5AINIQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811676625 (13h53m5.036859673s) [noise=true , address=77.90.31.11:4554 , piece_id=GENPAW6S7BKJ5AZJCXPAJTVF4SOTL5UA4T4E3GPP4C6BH36EE5CA, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811791101 (13h53m5.036974148s) [noise=true , address=45.91.116.10:4081 , piece_id=TBKB4HNCUD7C44E5VJRPIDYXX4RBDYCYZ3QAAF4KN7TUSFOZB74A, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.811928379 (13h53m5.037111427s) [noise=true , address=45.91.116.8:4045 , piece_id=BOBGOSJ5RZP7BLVDKQQ6E255QLUQTJLAOK3AFK2PSEFR225TJ2XQ, seg_pos={0 0}, upload=12180]: Uploading piece... Aug 26 05:56:49.882896864 (13h53m5.108079921s) [upload=12180]: Next returned nil segment Aug 26 05:56:49.882907514 (13h53m5.108090571s) [upload=12180]: Waiting for error group managing segments... Aug 26 05:56:49.893120667 (13h53m5.118303714s) [noise=true , address=45.91.116.10:4081 , piece_id=TBKB4HNCUD7C44E5VJRPIDYXX4RBDYCYZ3QAAF4KN7TUSFOZB74A, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.893997816 (13h53m5.119180874s) [noise=true , address=45.91.116.14:4190 , piece_id=SDU2YXVNCAXKUZD27FXV5QMQKELZMB6JFXPA63BK6N4JUONFXERA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894143190 (13h53m5.119326237s) [noise=true , address=45.91.116.14:4189 , piece_id=6WH5SDDZRHSRR6N623JHSVZFRDIPJ7GCBBTGOB6AMRQ6GH4FDVNQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894396877 (13h53m5.119579934s) [noise=true , address=45.91.116.8:4045 , piece_id=BOBGOSJ5RZP7BLVDKQQ6E255QLUQTJLAOK3AFK2PSEFR225TJ2XQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894549414 (13h53m5.119732461s) [noise=true , address=77.90.31.21:4877 , piece_id=5CI3JSRNRSDIZAWWHFRIIHJWZQV4BYYPCBNCVOYTYS2FTJCQKDZQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894736436 (13h53m5.119919493s) [noise=true , address=45.91.116.12:4225 , piece_id=I3G7OLIULCH4YQO62WU7YKBHYMC5VVQXAULCH5UQCOD7U3E2HD3Q, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.894993890 (13h53m5.120176937s) [noise=true , address=77.90.31.16:4735 , piece_id=OSQLWS6BGSBR2RGF2OAISHGDG3VPGC3UGFVL2WLPZMLQYO5AINIQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.895078880 (13h53m5.120261937s) [noise=true , address=45.91.116.5:4297 , piece_id=6EUXSCDMS2XPFBX465EGCMJZFZH2O2RZQISTV2XODEPKHTVDOMOQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.895359267 (13h53m5.120542314s) [noise=true , address=45.91.116.6:4975 , piece_id=NGMDRK6UBQBA25W7QUYA32DR54EQNRFKOGYHMCU6VCW2523QPGPA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.895424841 (13h53m5.120607898s) [noise=true , address=77.90.31.21:4879 , piece_id=JNTF36FINGPPYDKR7OV2GOLMFFHZUZV2TNK5VVLFVCDT3IUSFGPQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.896055406 (13h53m5.121238454s) [noise=true , address=45.91.116.4:4938 , piece_id=U2NTGDYIBJGM4C2PH3VXXDLFMPAVOYH7ZEFL4DGNH27GMECXLRVA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.897951745 (13h53m5.123134791s) [noise=true , address=77.90.31.13:4628 , piece_id=PXADU2MH7TXQU57ZQWMGSLYJ2SHP6YPH6UBOJFK4KPHXNBRLY6DQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.898959220 (13h53m5.124142267s) [noise=true , address=77.90.31.11:4554 , piece_id=GENPAW6S7BKJ5AZJCXPAJTVF4SOTL5UA4T4E3GPP4C6BH36EE5CA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: <nil> Aug 26 05:56:49.898974528 (13h53m5.124157576s) [seg_pos={0 0}, upload=12180]: Segment reached optimal threshold of 13 pieces. Aug 26 05:56:49.899042105 (13h53m5.124225152s) [noise=true , address=104.36.82.8:4969 , piece_id=I4F3Q7CAMFKBEYGX3GSZ5SSQSYHADMRBFTGC6TWSWN3HDAC3Y2YQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:49.899080618 (13h53m5.124263666s) [noise=true , address=77.90.31.18:4805 , piece_id=S5WGLC2RQOZPNSAGNQSVSOODCR7KJ46MGMWOQDPMPGT5CDNVRCKQ, seg_pos={0 0}, upload=12180]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:49.899108991 (13h53m5.124292039s) [noise=true , address=45.91.116.12:4226 , piece_id=SRNYOYHMUBOCNMLDATC4EUPPPBWSINM2U3NSFNCUJXRP7ZQIBVMA, seg_pos={0 0}, upload=12180]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:49.899133838 (13h53m5.124316895s) [seg_pos={0 0}, upload=12180]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:56:49.899232344 (13h53m5.124415401s) [upload=12180]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:49.899236652 (13h53m5.124419709s) [upload=12180]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:50.107204433 (13h53m5.332387509s) [upload=12180]: Done uploading segments... Aug 26 05:56:50.107285776 (13h53m5.332468852s) [upload=12180]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 431.014584ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [.X.X..XX.....X...................X.........XX...............] duration: 19.116025382s FAIL (2025-08-26 05:56:50 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:19.122249242 (13h42m34.347432309s) [upload=12027]: Starting upload Aug 26 05:46:19.122336606 (13h42m34.347519673s) [upload=12027]: Done starting upload Aug 26 05:46:19.122389295 (13h42m34.347572352s) [upload=12027]: Uploading segments... Aug 26 05:46:19.122426304 (13h42m34.347609352s) [upload=12027]: Got next segment. Inline: true Aug 26 05:46:19.122431003 (13h42m34.34761406s) [upload=12027]: Waiting for error group managing segments... Aug 26 05:46:19.122482891 (13h42m34.347665938s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:19.122487078 (13h42m34.347670135s) [upload=12027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:19.122489442 (13h42m34.3476725s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:30.124196038 (13h42m45.349379106s) [upload=12027]: Done uploading segments... Aug 26 05:46:30.124211648 (13h42m45.349394705s) [upload=12027]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 19.116051s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [.X...............X..X.........XX.........X.XX.X.............] duration: 5.183110332s remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:56:49 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 5.183125692s edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XX....XXX......X....X...X......XX.X.XX....XX.XX.........X.XX] duration: 2.714618819s FAIL (2025-08-26 05:56:48 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3B81C9C1F624, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 2.714643775s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [..X...............X..X.........XX.........X.XX.X............] duration: 23.6568948s remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:56:44 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 23.656907133s edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 16 out of 60 is failed (threshold=3) [XX..XXX....XX......XX..XX....X............XXX..............X] duration: 2.64141569s FAIL (2025-08-26 05:56:30 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3B7D7DD5E7BA, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 2.641438073s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 11 out of 60 is failed (threshold=3) [..XX.XX....XX.................X...XX........X.............X.] duration: 1.367305154s remote: "tcp:45.38.24.2:443" response headers: X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:56:12 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Content-Length=433 Server=Storj Vary=Origin X-Amz-Request-Id=185F3B796D1B7776 X-Request-Id=JvmnWJy1rYQ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:56:28 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B796D1B7776</RequestId><HostId></HostId></Error>" Duration: 1.367329981s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [.X.XX....X..X..X..X.......XX.....XXXXX.....XX.............X.] duration: 23.769003645s FAIL (2025-08-26 05:56:26 UTC) operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 23.769022932s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [...X...............X..X.........XX.........X.XX.X...........] duration: 36.751034846s remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:56:20 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 36.751055203s edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [X.X..X.............XX...............XX........X.............] duration: 13.917878538s FAIL (2025-08-26 05:56:17 UTC) operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 13.917914777s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXX....XXX......X....X...X......XX.X.XX....XX.XX.........X.X] duration: 15.65790751s FAIL (2025-08-26 05:56:15 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3B7A00DB44E3, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 15.657930995s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [X.XX.XXXXX....X..X...X...X......X..XX.X.......XX..........XX] duration: 29.887046485s FAIL (2025-08-26 05:56:14 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:55:44.664746853 (13h51m59.88992991s) [upload=12161]: Starting upload Aug 26 05:55:44.664828647 (13h51m59.890011704s) [upload=12161]: Done starting upload Aug 26 05:55:44.664907645 (13h51m59.890090692s) [upload=12161]: Uploading segments... Aug 26 05:55:44.665057537 (13h51m59.890240594s) [upload=12161]: Got next segment. Inline: false Aug 26 05:55:44.665090740 (13h51m59.890273797s) [upload=12161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:55:44.665094658 (13h51m59.890277704s) [upload=12161]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:55:48.682956388 (13h52m3.908139435s) [seg_pos={0 0}, upload=12161]: Begin upload segment... Aug 26 05:55:48.682988559 (13h52m3.908171595s) [seg_pos={0 0}, upload=12161]: Done begin upload segment. Aug 26 05:55:48.683007003 (13h52m3.90819005s) [noise=true , address=77.90.31.11:4554 , piece_id=N5IYBOD3U4STJWRRM64ZIUSQE5G36OOOQ2TDY2M2BQMQ45JU3R5Q, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.683082095 (13h52m3.908265172s) [noise=true , address=77.90.31.14:4661 , piece_id=AHEULGNGHJU4OTKMEGLJPDBPJ4CBA5QAW7RMIDF7GFTC64U345UQ, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.683307278 (13h52m3.908490326s) [noise=true , address=45.91.116.12:4226 , piece_id=Z2F3BQFOBYW2RHHHXCKXJSKDMTJC6EOFIOJSKO5XNUUFXAKXHPKA, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.683515169 (13h52m3.908698216s) [noise=true , address=77.90.31.15:4699 , piece_id=47IJXV2E7FLA2BM53YEZTOQRFLNMS7L6LWMO3N45Q724BDMJK6NQ, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.683590832 (13h52m3.908773909s) [noise=true , address=77.90.31.13:4627 , piece_id=TBC7QOMMTOLDHHOK66BHHDH4HCYAUDNSK7NABWHAEVNZ4K43GAAA, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.683697392 (13h52m3.908880439s) [noise=true , address=45.91.116.9:4263 , piece_id=54K3PIPMN7SPGTSN6TSV7AKACXGZQVAHHKFSGONZMBWUG4LSJCFQ, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.683893140 (13h52m3.909076187s) [noise=true , address=45.91.116.8:4045 , piece_id=QBCQW5YSNF763N6OW3RNSN6OJ34KN7TG3FMGNBOZDLESE7FOOKUQ, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684005271 (13h52m3.909188338s) [noise=true , address=45.91.116.10:4081 , piece_id=JBDJ4TR2EDNWRBFWE6CFHNAAE4DX2SGFDAI2ASP6P6VTIPNPE77Q, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684122002 (13h52m3.909305048s) [noise=true , address=77.90.31.14:4662 , piece_id=BLMYX5GFLPGH47Y2ME7AWQXS5K4PQU2G7UTLMWXXGRZDRAYHZDDQ, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684348857 (13h52m3.909531905s) [noise=true , address=45.91.116.12:4225 , piece_id=QSBOLRJLKEUXNSIAIOV47Y3QK2ZGWNBFDR5U5UIEWVV7QLVWC7EA, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684413859 (13h52m3.909596937s) [noise=true , address=45.91.116.8:4048 , piece_id=PAHHFQJ4URR3LO3D4NPEUHVUQC4V6YVHMLEIF7A4LD6X2NAVOUDQ, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684532112 (13h52m3.90971516s) [noise=true , address=77.90.31.12:4592 , piece_id=45QV75YZXPR7WUTLFOPU773DNER7QFKTPVRYXAKMPW24YOVTJB7A, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684736136 (13h52m3.909919183s) [noise=true , address=45.91.116.11:4118 , piece_id=FDJLM6VQE4O7LONGIABGTX22EULKXVZ74F2A3S27DIGAKRMBSAGA, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.684759309 (13h52m3.909942387s) [noise=true , address=45.91.116.11:4119 , piece_id=BEX3FU2ZL4BN7FEWPIV36BMH47B4GN6MOOPAERGQ6CY6472OTZ5A, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.685305357 (13h52m3.910488433s) [noise=true , address=77.90.31.16:4735 , piece_id=QRYXP4ZK3MENKMY352D5DIWSCLMVT4D4IIXRZKKY5JGCPCROQIAA, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.686014290 (13h52m3.911197357s) [noise=true , address=45.91.116.7:4009 , piece_id=3UDGRDOKKVZCQO6ONDKI44C26DCESQQ74VQXKPU5NYFU5XAZWGMA, seg_pos={0 0}, upload=12161]: Uploading piece... Aug 26 05:55:48.767131206 (13h52m3.992314253s) [upload=12161]: Next returned nil segment Aug 26 05:55:48.767142888 (13h52m3.992325925s) [upload=12161]: Waiting for error group managing segments... Aug 26 05:55:48.774827094 (13h52m4.000010141s) [noise=true , address=45.91.116.12:4226 , piece_id=Z2F3BQFOBYW2RHHHXCKXJSKDMTJC6EOFIOJSKO5XNUUFXAKXHPKA, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.774874753 (13h52m4.0000578s) [noise=true , address=45.91.116.7:4009 , piece_id=3UDGRDOKKVZCQO6ONDKI44C26DCESQQ74VQXKPU5NYFU5XAZWGMA, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.775113261 (13h52m4.000296309s) [noise=true , address=45.91.116.8:4045 , piece_id=QBCQW5YSNF763N6OW3RNSN6OJ34KN7TG3FMGNBOZDLESE7FOOKUQ, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.775280977 (13h52m4.000464024s) [noise=true , address=45.91.116.11:4119 , piece_id=BEX3FU2ZL4BN7FEWPIV36BMH47B4GN6MOOPAERGQ6CY6472OTZ5A, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.775551977 (13h52m4.000735024s) [noise=true , address=45.91.116.12:4225 , piece_id=QSBOLRJLKEUXNSIAIOV47Y3QK2ZGWNBFDR5U5UIEWVV7QLVWC7EA, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.775581663 (13h52m4.00076471s) [noise=true , address=45.91.116.10:4081 , piece_id=JBDJ4TR2EDNWRBFWE6CFHNAAE4DX2SGFDAI2ASP6P6VTIPNPE77Q, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.775804001 (13h52m4.000987048s) [noise=true , address=45.91.116.9:4263 , piece_id=54K3PIPMN7SPGTSN6TSV7AKACXGZQVAHHKFSGONZMBWUG4LSJCFQ, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.776429258 (13h52m4.001612305s) [noise=true , address=45.91.116.11:4118 , piece_id=FDJLM6VQE4O7LONGIABGTX22EULKXVZ74F2A3S27DIGAKRMBSAGA, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.776654792 (13h52m4.001837838s) [noise=true , address=45.91.116.8:4048 , piece_id=PAHHFQJ4URR3LO3D4NPEUHVUQC4V6YVHMLEIF7A4LD6X2NAVOUDQ, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.777827528 (13h52m4.003010575s) [noise=true , address=77.90.31.11:4554 , piece_id=N5IYBOD3U4STJWRRM64ZIUSQE5G36OOOQ2TDY2M2BQMQ45JU3R5Q, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.778107353 (13h52m4.003290401s) [noise=true , address=77.90.31.14:4661 , piece_id=AHEULGNGHJU4OTKMEGLJPDBPJ4CBA5QAW7RMIDF7GFTC64U345UQ, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.778385307 (13h52m4.003568354s) [noise=true , address=77.90.31.15:4699 , piece_id=47IJXV2E7FLA2BM53YEZTOQRFLNMS7L6LWMO3N45Q724BDMJK6NQ, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.778679281 (13h52m4.003862327s) [noise=true , address=77.90.31.14:4662 , piece_id=BLMYX5GFLPGH47Y2ME7AWQXS5K4PQU2G7UTLMWXXGRZDRAYHZDDQ, seg_pos={0 0}, upload=12161]: Done uploading piece. err: <nil> Aug 26 05:55:48.778684300 (13h52m4.003867347s) [seg_pos={0 0}, upload=12161]: Segment reached optimal threshold of 13 pieces. Aug 26 05:55:48.778760122 (13h52m4.003943169s) [noise=true , address=77.90.31.16:4735 , piece_id=QRYXP4ZK3MENKMY352D5DIWSCLMVT4D4IIXRZKKY5JGCPCROQIAA, seg_pos={0 0}, upload=12161]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: expected piece hash; context canceled; EOF Aug 26 05:55:48.778823511 (13h52m4.004006548s) [noise=true , address=77.90.31.13:4627 , piece_id=TBC7QOMMTOLDHHOK66BHHDH4HCYAUDNSK7NABWHAEVNZ4K43GAAA, seg_pos={0 0}, upload=12161]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Aug 26 05:55:48.778871332 (13h52m4.004054379s) [noise=true , address=77.90.31.12:4592 , piece_id=45QV75YZXPR7WUTLFOPU773DNER7QFKTPVRYXAKMPW24YOVTJB7A, seg_pos={0 0}, upload=12161]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Aug 26 05:55:48.778878845 (13h52m4.004061893s) [seg_pos={0 0}, upload=12161]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:55:48.778919252 (13h52m4.004102298s) [upload=12161]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:55:48.778922738 (13h52m4.004105785s) [upload=12161]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:10.347153866 (13h52m25.572337003s) [upload=12161]: Done uploading segments... Aug 26 05:56:10.347242342 (13h52m25.572425429s) [upload=12161]: Upload finished. err: <nil> Duration: 29.887066333s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 11 out of 60 is failed (threshold=3) [...XX.XX....XX.................X...XX........X.............X] duration: 254.064553ms remote: "tcp:45.38.24.2:443" response headers: X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:56:12 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Content-Length=433 Server=Storj Vary=Origin X-Amz-Request-Id=185F3B796D1B7776 X-Request-Id=JvmnWJy1rYQ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:56:12 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B796D1B7776</RequestId><HostId></HostId></Error>" Duration: 254.096674ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 11 out of 60 is failed (threshold=3) [..XX.X.X.X.....X..X......................X.....X..X........X] duration: 489.271748ms FAIL (2025-08-26 05:56:08 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 489.301093ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 26 out of 60 is failed (threshold=3) [XXX.XX.......X..XX.XXXXXXX.....XX..X.XXX......XX.X........XX] duration: 15.175211995s FAIL (2025-08-26 05:56:02 UTC) operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 15.17523083s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [..X.XX....X..X..X..X.......XX.....XXXXX.....XX.............X] duration: 28.094389139s FAIL (2025-08-26 05:56:02 UTC) operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 28.094410389s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [..X.X..XX.....X...................X.........XX..............] duration: 3.576821836s FAIL (2025-08-26 05:55:56 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:19.122249242 (13h42m34.347432309s) [upload=12027]: Starting upload Aug 26 05:46:19.122336606 (13h42m34.347519673s) [upload=12027]: Done starting upload Aug 26 05:46:19.122389295 (13h42m34.347572352s) [upload=12027]: Uploading segments... Aug 26 05:46:19.122426304 (13h42m34.347609352s) [upload=12027]: Got next segment. Inline: true Aug 26 05:46:19.122431003 (13h42m34.34761406s) [upload=12027]: Waiting for error group managing segments... Aug 26 05:46:19.122482891 (13h42m34.347665938s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:19.122487078 (13h42m34.347670135s) [upload=12027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:19.122489442 (13h42m34.3476725s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:30.124196038 (13h42m45.349379106s) [upload=12027]: Done uploading segments... Aug 26 05:46:30.124211648 (13h42m45.349394705s) [upload=12027]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 3.576851344s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 4 out of 60 is failed (threshold=6) [.X....X....X.......................X........................] duration: 8.039868612s remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD X-Request-Id=ZFwx4y7eAxE Content-Type=text/html; charset=utf-8 Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:38:38 GMT target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:55:48 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 8.039900012s edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.XXX....XXX......X....X...X......XX.X.XX....XX.XX.........X.] duration: 20.522000744s FAIL (2025-08-26 05:55:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.522021383s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [.XX..XXX....XX......XX..XX....X............XXX..............] duration: 23.335704932s FAIL (2025-08-26 05:55:44 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.335727745s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 16 out of 60 is failed (threshold=3) [.XX.X.X.XX....X..X...XX.XXX..........X..XX..................] duration: 12.881885982s FAIL (2025-08-26 05:55:41 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:40:50.174916229 (13h37m5.400099286s) [upload=11948]: Starting upload Aug 26 05:40:50.174950314 (13h37m5.400133371s) [upload=11948]: Done starting upload Aug 26 05:40:50.174991591 (13h37m5.400174638s) [upload=11948]: Uploading segments... Aug 26 05:40:50.175014524 (13h37m5.400197571s) [upload=11948]: Got next segment. Inline: true Aug 26 05:40:50.175017720 (13h37m5.400200767s) [upload=11948]: Waiting for error group managing segments... Aug 26 05:40:50.175043148 (13h37m5.400226195s) [upload=11948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:50.175046144 (13h37m5.400229191s) [upload=11948]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:50.175048268 (13h37m5.400231315s) [upload=11948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:41:09.312511333 (13h37m24.53769439s) [upload=11948]: Done uploading segments... Aug 26 05:41:09.312532192 (13h37m24.537715249s) [upload=11948]: Upload finished. err: <nil> Duration:* 12.881901291s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [....XX.XX....XX.................X...XX........X.............] duration: 3.634874682s remote: "tcp:45.38.24.2:443" response headers: Content-Type=application/xml Server=Storj X-Amz-Request-Id=185F3AEE0EB33B6A X-Request-Id=McQuPBREAb1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:46:14 GMT Accept-Ranges=bytes Content-Length=433 Content-Security-Policy=block-all-mixed-content Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:55:41 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AEE0EB33B6A</RequestId><HostId></HostId></Error>" Duration: 3.634939914s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [...XX.X.X.X.....X..X......................X.....X..X........] duration: 46.8896789s FAIL (2025-08-26 05:55:39 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== O4JMKFA5RN4IQN2CRCW5VBPYXLMYFOX3VA7ICGQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.337676887 (13h43m30.562859964s) [noise=true , address=77.90.31.15:4717 , piece_id=5I67NYFJRZT7ORPYPVSJA56BB4LYVB3WUQOE7ZXFFHTCHMI2TYVQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.338323312 (13h43m30.563506388s) [noise=true , address=104.36.82.44:4890 , piece_id=NYLN3X5UXITCKCFZ6E2WIGI7WZV7USAFIEMUNESNBYU4BVOKAW5Q, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.340398391 (13h43m30.565581469s) [noise=true , address=79.127.205.245:4220 , piece_id=FPXMKCW6YUM5CGTJ27UUGSXZE7VZ6MS542O3H4NYHAKYPM5U4OFQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.340778866 (13h43m30.565961953s) [noise=true , address=104.36.82.60:4365 , piece_id=IOYRFBESQVMDVOEH677WCO6CDSU6LOMQOZQOMMXT2QSEAXDRLFWA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.341147428 (13h43m30.566330505s) [noise=true , address=79.127.205.246:4284 , piece_id=EC3P5P5T2ODWFBH4VQ3666AR3TSW3LJZKZU7P2UFB76V22RD5VQQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.341604627 (13h43m30.566787705s) [noise=true , address=104.36.82.11:4128 , piece_id=QIDJJMBP36RUA4XXQ2Q2HDF2R4DW53A222OEQEWA2AMX6RSQCVFA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.341967629 (13h43m30.567150706s) [noise=true , address=77.90.31.18:4821 , piece_id=RSEEETAXPX2UNFT3NAGVQIDD6HZXMX2BRHMKMTVXPNWVXKFVSYXQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.342312617 (13h43m30.567495705s) [noise=true , address=45.91.116.6:4002 , piece_id=42SYNFZV4JY4CHJ4KHGEPLK5BY3PVIII66CTFE4I4SAJLVELULYQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.342682082 (13h43m30.56786516s) [noise=true , address=104.36.82.5:4848 , piece_id=YPQNRHC7BMQET4FH3IOP3WVAY6D3H5FCVJSSIYWL6KAJ43GFPDWA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.343107501 (13h43m30.568290578s) [noise=true , address=79.127.219.47:4065 , piece_id=J774Z6CPOJBLVO4KC2ODC2FMFRG5LQYLHL2LFQF3AR4SWD2CD6BQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.343696528 (13h43m30.568879605s) [noise=true , address=104.36.82.23:4466 , piece_id=U6DXP2ADNBE657QXHKT633JIV66ROOTRQKF4B7OAJTVDOV3CRLUA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.347700131 (13h43m30.572883208s) [noise=true , address=104.36.82.11:4140 , piece_id=BA5JUOEPAD4TNNCFEHMQ5KO3CDJJ5L56GDXZMT6VIGGBXCETBYGQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.353294242 (13h43m30.578477318s) [noise=true , address=79.127.205.234:4619 , piece_id=KURMGY53IW7WE6SMXSLZDFDMO6SSSILYL66NBM54LRPAZ3HVZXQA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.357005705 (13h43m30.582188852s) [noise=true , address=104.36.82.50:4017 , piece_id=VXMNPE5WHITPQP7LQUDAEYAN6LYYWPWIR6BTE2FJXECSXKQAHVWQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.360097525 (13h43m30.585280602s) [noise=true , address=64.49.13.154:4934 , piece_id=KDCFYU63FXZ4BHOYEGGO5VC6M3KC7JMERNSY5F24HM7TWJ2PRVHA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.360718501 (13h43m30.585901589s) [noise=true , address=77.90.31.16:4752 , piece_id=FKJQ66EGGM57CJOCPKD4V4K4YGWGRQJTFVA5JK2YZBOUFO2KFQDA, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.363118261 (13h43m30.588301337s) [noise=true , address=77.90.31.19:4855 , piece_id=NCMORUSZ6XAEPRPT4YPXHWIZGP5BVLWCDCNBTOC7ECK6QMU22WFQ, seg_pos={0 0}, upload=12041]: Uploading piece... Aug 26 05:47:15.442534509 (13h43m30.667717586s) [upload=12041]: Next returned nil segment Aug 26 05:47:15.442550258 (13h43m30.667733345s) [upload=12041]: Waiting for error group managing segments... Aug 26 05:47:15.451182057 (13h43m30.676365164s) [noise=true , address=77.90.31.18:4821 , piece_id=RSEEETAXPX2UNFT3NAGVQIDD6HZXMX2BRHMKMTVXPNWVXKFVSYXQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.452462604 (13h43m30.677645681s) [noise=true , address=79.127.219.47:4065 , piece_id=J774Z6CPOJBLVO4KC2ODC2FMFRG5LQYLHL2LFQF3AR4SWD2CD6BQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.454260442 (13h43m30.679443529s) [noise=true , address=45.91.116.6:4002 , piece_id=42SYNFZV4JY4CHJ4KHGEPLK5BY3PVIII66CTFE4I4SAJLVELULYQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.455280057 (13h43m30.680463144s) [noise=true , address=77.90.31.15:4717 , piece_id=5I67NYFJRZT7ORPYPVSJA56BB4LYVB3WUQOE7ZXFFHTCHMI2TYVQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.456937692 (13h43m30.682120769s) [noise=true , address=104.36.82.5:4848 , piece_id=YPQNRHC7BMQET4FH3IOP3WVAY6D3H5FCVJSSIYWL6KAJ43GFPDWA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.457694363 (13h43m30.68287745s) [noise=true , address=104.36.82.11:4128 , piece_id=QIDJJMBP36RUA4XXQ2Q2HDF2R4DW53A222OEQEWA2AMX6RSQCVFA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.458098192 (13h43m30.68328127s) [noise=true , address=104.36.82.11:4140 , piece_id=BA5JUOEPAD4TNNCFEHMQ5KO3CDJJ5L56GDXZMT6VIGGBXCETBYGQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.458479239 (13h43m30.683662315s) [noise=true , address=104.36.82.23:4466 , piece_id=U6DXP2ADNBE657QXHKT633JIV66ROOTRQKF4B7OAJTVDOV3CRLUA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.458886112 (13h43m30.684069199s) [noise=true , address=104.36.82.44:4890 , piece_id=NYLN3X5UXITCKCFZ6E2WIGI7WZV7USAFIEMUNESNBYU4BVOKAW5Q, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.459042417 (13h43m30.684225493s) [noise=true , address=104.36.82.50:4017 , piece_id=VXMNPE5WHITPQP7LQUDAEYAN6LYYWPWIR6BTE2FJXECSXKQAHVWQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.459511177 (13h43m30.684694254s) [noise=true , address=104.36.82.60:4365 , piece_id=IOYRFBESQVMDVOEH677WCO6CDSU6LOMQOZQOMMXT2QSEAXDRLFWA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.468110394 (13h43m30.693293461s) [noise=true , address=77.90.31.19:4855 , piece_id=NCMORUSZ6XAEPRPT4YPXHWIZGP5BVLWCDCNBTOC7ECK6QMU22WFQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.620114931 (13h43m30.845298009s) [noise=true , address=79.127.201.215:4349 , piece_id=5KA4X7E475FZVO4JMKFA5RN4IQN2CRCW5VBPYXLMYFOX3VA7ICGQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.663559442 (13h43m30.888742509s) [noise=true , address=77.90.31.16:4752 , piece_id=FKJQ66EGGM57CJOCPKD4V4K4YGWGRQJTFVA5JK2YZBOUFO2KFQDA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: <nil> Aug 26 05:47:15.663591131 (13h43m30.888774198s) [seg_pos={0 0}, upload=12041]: Segment reached optimal threshold of 14 pieces. Aug 26 05:47:15.663955185 (13h43m30.889138262s) [noise=true , address=79.127.205.245:4220 , piece_id=FPXMKCW6YUM5CGTJ27UUGSXZE7VZ6MS542O3H4NYHAKYPM5U4OFQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123xj8bTNShfvqNJh3mSvM6V4TeXQoxFHR5TKELoUKTJJTzDyjD): protocol: write tcp 5.161.89.252:33592->79.127.205.245:4220: use of closed network connection; write tcp 5.161.89.252:33592->79.127.205.245:4220: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33592->79.127.205.245:4220: use of closed network connection Aug 26 05:47:15.663980523 (13h43m30.88916359s) [noise=true , address=79.127.205.246:4284 , piece_id=EC3P5P5T2ODWFBH4VQ3666AR3TSW3LJZKZU7P2UFB76V22RD5VQQ, seg_pos={0 0}, upload=12041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1z7gUykS9V5o6wvXSCmK5jBvXK6XAqck4Z348Ryz97PLEyVtD2): protocol: write tcp 5.161.89.252:61910->79.127.205.246:4284: use of closed network connection; write tcp 5.161.89.252:61910->79.127.205.246:4284: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61910->79.127.205.246:4284: use of closed network connection Aug 26 05:47:15.664038241 (13h43m30.889221308s) [noise=true , address=64.49.13.154:4934 , piece_id=KDCFYU63FXZ4BHOYEGGO5VC6M3KC7JMERNSY5F24HM7TWJ2PRVHA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TEiTVsNxZYEYXCnfZ8h4UiEqCBRBPih3YP5cnEZEMTYoiPEqC): protocol: read tcp 5.161.89.252:62488->64.49.13.154:4934: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:47:15.664095149 (13h43m30.889278225s) [noise=true , address=79.127.205.234:4619 , piece_id=KURMGY53IW7WE6SMXSLZDFDMO6SSSILYL66NBM54LRPAZ3HVZXQA, seg_pos={0 0}, upload=12041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yrNdQVXCpM81T6PiYFQKrHiesKtqcQAAMExwqxTeQsSqH6BAK): protocol: write tcp 5.161.89.252:33526->79.127.205.234:4619: use of closed network connection; write tcp 5.161.89.252:33526->79.127.205.234:4619: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33526->79.127.205.234:4619: use of closed network connection Aug 26 05:47:15.664159890 (13h43m30.889342967s) [seg_pos={0 0}, upload=12041]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:47:15.664249217 (13h43m30.889432295s) [upload=12041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:47:15.664255018 (13h43m30.889438096s) [upload=12041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:26.126651265 (13h43m41.351834322s) [upload=12041]: Done uploading segments... Aug 26 05:47:26.126673085 (13h43m41.351856142s) [upload=12041]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 46.889716641s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [...X.X..XX.....X...................X.........XX.............] duration: 17.713355274s FAIL (2025-08-26 05:55:37 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:19.122249242 (13h42m34.347432309s) [upload=12027]: Starting upload Aug 26 05:46:19.122336606 (13h42m34.347519673s) [upload=12027]: Done starting upload Aug 26 05:46:19.122389295 (13h42m34.347572352s) [upload=12027]: Uploading segments... Aug 26 05:46:19.122426304 (13h42m34.347609352s) [upload=12027]: Got next segment. Inline: true Aug 26 05:46:19.122431003 (13h42m34.34761406s) [upload=12027]: Waiting for error group managing segments... Aug 26 05:46:19.122482891 (13h42m34.347665938s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:19.122487078 (13h42m34.347670135s) [upload=12027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:19.122489442 (13h42m34.3476725s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:30.124196038 (13h42m45.349379106s) [upload=12027]: Done uploading segments... Aug 26 05:46:30.124211648 (13h42m45.349394705s) [upload=12027]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 17.713382816s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 17 out of 60 is failed (threshold=3) [X..X.XX....X..X..X..X.......XX.....XXXXX.....XX.............] duration: 16.608903825s FAIL (2025-08-26 05:55:25 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 16.608928202s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [.X.X..X.............XX...............XX........X............] duration: 17.758925888s FAIL (2025-08-26 05:55:20 UTC) operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 17.758979079s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 26 out of 60 is failed (threshold=3) [XXXX.XX.......X..XX.XXXXXXX.....XX..X.XXX......XX.X........X] duration: 30.020513176s FAIL (2025-08-26 05:55:19 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.020532342s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [..XXX....XXX......X....X...X......XX.X.XX....XX.XX.........X] duration: 30.001257882s FAIL (2025-08-26 05:55:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001309058s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.X.XX.XXXXX....X..X...X...X......X..XX.X.......XX..........X] duration: 30.001426378s FAIL (2025-08-26 05:55:12 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:54:42.872122017 (13h50m58.097305095s) [upload=12146]: Starting upload Aug 26 05:54:42.872213659 (13h50m58.097396736s) [upload=12146]: Done starting upload Aug 26 05:54:42.872346399 (13h50m58.097529477s) [upload=12146]: Uploading segments... Aug 26 05:54:42.872649770 (13h50m58.097832847s) [upload=12146]: Got next segment. Inline: false Aug 26 05:54:42.872690587 (13h50m58.097873664s) [upload=12146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:54:42.872696147 (13h50m58.097879224s) [upload=12146]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:54:43.013724789 (13h50m58.238907877s) [seg_pos={0 0}, upload=12146]: Begin upload segment... Aug 26 05:54:43.013789530 (13h50m58.238972608s) [seg_pos={0 0}, upload=12146]: Done begin upload segment. Aug 26 05:54:43.013806863 (13h50m58.23899001s) [noise=true , address=77.90.31.13:4628 , piece_id=XBZKVKC6JFO5NASZPUKJW2V44YSOYS2B36P4PTGE5UFRV5O4THPQ, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.013817503 (13h50m58.23900058s) [noise=true , address=77.90.31.15:4698 , piece_id=C5MGQ5TMLRYW52T3OS4MXG76FTJ2NTASQVMSWYOQSUJ3BD62PUFA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.013955182 (13h50m58.23913827s) [noise=true , address=45.91.116.4:4938 , piece_id=L7L54HKOHRGYFCVZCGXUHVW7WU6CBPJILIYBRRM33SDR7ZPL5CDA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.014506469 (13h50m58.239689556s) [noise=true , address=45.91.116.11:4118 , piece_id=PQQVK4N6FI64JPD66VUCDCCHAOTEOPETWTAEO7LUFSYOWRYRNDUA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.014710252 (13h50m58.23989334s) [noise=true , address=45.91.116.11:4119 , piece_id=6OKWUIX2ICI6S37E4DT7CA565LHPYSDG4U56VRKJ7EY7HTOGUFDA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.015099645 (13h50m58.240282732s) [noise=true , address=45.91.116.5:4299 , piece_id=VHUT22NWXKJITQF2IUFOAZMYADKWGXRDGEWILMUBV5DYJPY5Q5TA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.015391154 (13h50m58.24057424s) [noise=true , address=45.91.116.8:4045 , piece_id=S4F3RYIY4EP22YHFMBOWTFEUW4U3FBNZVORJZL53GAYW3UCBNYMA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.015708601 (13h50m58.240891688s) [noise=true , address=77.90.31.13:4627 , piece_id=RHBUV3SPBGYINCILXNMUQ2MQZSNSMEVKPGSXQDWEMMAND4NZK5CA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.016066224 (13h50m58.24124931s) [noise=true , address=45.91.116.5:4298 , piece_id=7RL5FAF6B5IN7GJYGDHXUFH5RXE272G2X6M3U2UJK3OZ53XINTAA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.016316855 (13h50m58.241499941s) [noise=true , address=77.90.31.11:4553 , piece_id=44QQJUBK2ZFL23S726YPZF74OZ4EUUAMGSRNTSYEOYYZXILVJKSA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.016647767 (13h50m58.241830864s) [noise=true , address=45.91.116.6:4975 , piece_id=EORXYIKRICPCMYU5UB42MZS44UVSX4P3U2CEF4OEZ5ZSE2SJTA4A, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.016887730 (13h50m58.242070816s) [noise=true , address=45.91.116.4:4937 , piece_id=LHKOSS6HZ4N2NPXEZLIPRH74SOOFM4B6BOAGDKVZKNI7GCQX57XA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.017304683 (13h50m58.242487781s) [noise=true , address=77.90.31.19:4841 , piece_id=NGW3CT4PNSF3QOPVOWBFRLSXLU3WLINDIZU775TVZCLT3EC4WCDA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.017502946 (13h50m58.242686033s) [noise=true , address=45.91.116.7:4010 , piece_id=UQLI4EVH6CHLAUZUC66TPXLYJ5THEHLGPD67ODURR7W4BPGEO53A, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.017953775 (13h50m58.243136861s) [noise=true , address=77.90.31.14:4663 , piece_id=MGJXPQXY4NVZHYSDS473D3CO7EOFOXVUTTQRSHVLLWD2PZEFNHGA, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.018554344 (13h50m58.243737431s) [noise=true , address=45.91.116.10:4083 , piece_id=MQD7YERYSV6GP5QXM5TTJVHUF2D5SXAMEMGAGRPS5S3JUXUUTLQQ, seg_pos={0 0}, upload=12146]: Uploading piece... Aug 26 05:54:43.118412155 (13h50m58.343595201s) [upload=12146]: Next returned nil segment Aug 26 05:54:43.118422684 (13h50m58.343605731s) [upload=12146]: Waiting for error group managing segments... Aug 26 05:54:43.126780056 (13h50m58.351963113s) [noise=true , address=45.91.116.6:4975 , piece_id=EORXYIKRICPCMYU5UB42MZS44UVSX4P3U2CEF4OEZ5ZSE2SJTA4A, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.126828056 (13h50m58.352011113s) [noise=true , address=77.90.31.11:4553 , piece_id=44QQJUBK2ZFL23S726YPZF74OZ4EUUAMGSRNTSYEOYYZXILVJKSA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.127361871 (13h50m58.352544917s) [noise=true , address=45.91.116.10:4083 , piece_id=MQD7YERYSV6GP5QXM5TTJVHUF2D5SXAMEMGAGRPS5S3JUXUUTLQQ, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.127476646 (13h50m58.352659703s) [noise=true , address=45.91.116.8:4045 , piece_id=S4F3RYIY4EP22YHFMBOWTFEUW4U3FBNZVORJZL53GAYW3UCBNYMA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.127707401 (13h50m58.352890447s) [noise=true , address=45.91.116.5:4298 , piece_id=7RL5FAF6B5IN7GJYGDHXUFH5RXE272G2X6M3U2UJK3OZ53XINTAA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.128417516 (13h50m58.353600564s) [noise=true , address=45.91.116.4:4938 , piece_id=L7L54HKOHRGYFCVZCGXUHVW7WU6CBPJILIYBRRM33SDR7ZPL5CDA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.128570324 (13h50m58.353753381s) [noise=true , address=77.90.31.13:4628 , piece_id=XBZKVKC6JFO5NASZPUKJW2V44YSOYS2B36P4PTGE5UFRV5O4THPQ, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.128733741 (13h50m58.353916788s) [noise=true , address=77.90.31.14:4663 , piece_id=MGJXPQXY4NVZHYSDS473D3CO7EOFOXVUTTQRSHVLLWD2PZEFNHGA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.129212432 (13h50m58.354395489s) [noise=true , address=45.91.116.11:4119 , piece_id=6OKWUIX2ICI6S37E4DT7CA565LHPYSDG4U56VRKJ7EY7HTOGUFDA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.130907310 (13h50m58.356090367s) [noise=true , address=77.90.31.15:4698 , piece_id=C5MGQ5TMLRYW52T3OS4MXG76FTJ2NTASQVMSWYOQSUJ3BD62PUFA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.131308154 (13h50m58.356491201s) [noise=true , address=77.90.31.13:4627 , piece_id=RHBUV3SPBGYINCILXNMUQ2MQZSNSMEVKPGSXQDWEMMAND4NZK5CA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.132632596 (13h50m58.357815653s) [noise=true , address=45.91.116.11:4118 , piece_id=PQQVK4N6FI64JPD66VUCDCCHAOTEOPETWTAEO7LUFSYOWRYRNDUA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.137166205 (13h50m58.362349262s) [noise=true , address=45.91.116.4:4937 , piece_id=LHKOSS6HZ4N2NPXEZLIPRH74SOOFM4B6BOAGDKVZKNI7GCQX57XA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: <nil> Aug 26 05:54:43.137182555 (13h50m58.362365603s) [seg_pos={0 0}, upload=12146]: Segment reached optimal threshold of 13 pieces. Aug 26 05:54:43.137269338 (13h50m58.362452395s) [noise=true , address=45.91.116.7:4010 , piece_id=UQLI4EVH6CHLAUZUC66TPXLYJ5THEHLGPD67ODURR7W4BPGEO53A, seg_pos={0 0}, upload=12146]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 26 05:54:43.137365639 (13h50m58.362548696s) [noise=true , address=45.91.116.5:4299 , piece_id=VHUT22NWXKJITQF2IUFOAZMYADKWGXRDGEWILMUBV5DYJPY5Q5TA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: write tcp 5.161.89.252:58730->45.91.116.5:4299: use of closed network connection; write tcp 5.161.89.252:58730->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58730->45.91.116.5:4299: use of closed network connection Aug 26 05:54:43.137402709 (13h50m58.362585766s) [noise=true , address=77.90.31.19:4841 , piece_id=NGW3CT4PNSF3QOPVOWBFRLSXLU3WLINDIZU775TVZCLT3EC4WCDA, seg_pos={0 0}, upload=12146]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 5.161.89.252:57994->77.90.31.19:4841: use of closed network connection; write tcp 5.161.89.252:57994->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:57994->77.90.31.19:4841: use of closed network connection Aug 26 05:54:43.137411556 (13h50m58.362594603s) [seg_pos={0 0}, upload=12146]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:54:43.137472399 (13h50m58.362655447s) [upload=12146]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:54:43.137476387 (13h50m58.362659444s) [upload=12146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:55:04.809336868 (13h51m20.034519945s) [upload=12146]: Done uploading segments... Aug 26 05:55:04.809356545 (13h51m20.034539622s) [upload=12146]: Upload finished. err: <nil> Duration: 30.00144857s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [....X...............X..X.........XX.........X.XX.X..........] duration: 10.318491539s remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:55:08 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 10.318517878s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [....X.X..XX.....X...................X.........XX............] duration: 12.169809399s FAIL (2025-08-26 05:55:05 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:19.122249242 (13h42m34.347432309s) [upload=12027]: Starting upload Aug 26 05:46:19.122336606 (13h42m34.347519673s) [upload=12027]: Done starting upload Aug 26 05:46:19.122389295 (13h42m34.347572352s) [upload=12027]: Uploading segments... Aug 26 05:46:19.122426304 (13h42m34.347609352s) [upload=12027]: Got next segment. Inline: true Aug 26 05:46:19.122431003 (13h42m34.34761406s) [upload=12027]: Waiting for error group managing segments... Aug 26 05:46:19.122482891 (13h42m34.347665938s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:19.122487078 (13h42m34.347670135s) [upload=12027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:19.122489442 (13h42m34.3476725s) [upload=12027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:30.124196038 (13h42m45.349379106s) [upload=12027]: Done uploading segments... Aug 26 05:46:30.124211648 (13h42m45.349394705s) [upload=12027]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 12.169836589s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 16 out of 60 is failed (threshold=3) [..XX.X.X.XX....X..X...XX.XXX..........X..XX.................] duration: 18.787137526s FAIL (2025-08-26 05:55:04 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:40:50.174916229 (13h37m5.400099286s) [upload=11948]: Starting upload Aug 26 05:40:50.174950314 (13h37m5.400133371s) [upload=11948]: Done starting upload Aug 26 05:40:50.174991591 (13h37m5.400174638s) [upload=11948]: Uploading segments... Aug 26 05:40:50.175014524 (13h37m5.400197571s) [upload=11948]: Got next segment. Inline: true Aug 26 05:40:50.175017720 (13h37m5.400200767s) [upload=11948]: Waiting for error group managing segments... Aug 26 05:40:50.175043148 (13h37m5.400226195s) [upload=11948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:50.175046144 (13h37m5.400229191s) [upload=11948]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:50.175048268 (13h37m5.400231315s) [upload=11948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:41:09.312511333 (13h37m24.53769439s) [upload=11948]: Done uploading segments... Aug 26 05:41:09.312532192 (13h37m24.537715249s) [upload=11948]: Upload finished. err: <nil> Duration:* 18.787155651s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [.....XX.XX....XX.................X...XX........X............] duration: 8.505036871s remote: "tcp:45.38.24.2:443" response headers: Content-Type=application/xml Server=Storj X-Amz-Request-Id=185F3AEE0EB33B6A X-Request-Id=McQuPBREAb1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:46:14 GMT Accept-Ranges=bytes Content-Length=433 Content-Security-Policy=block-all-mixed-content Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:55:02 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AEE0EB33B6A</RequestId><HostId></HostId></Error>" Duration: 8.505067057s edge gateway vhost us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)