Orbiter History

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 47.744727ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:24:48 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 47.769604ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 443.047888ms FAIL (2025-08-27 21:24:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:18.742567354 (3h23m41.972052711s) [upload=3517]: Starting upload Aug 27 21:02:18.742644770 (3h23m41.972130117s) [upload=3517]: Done starting upload Aug 27 21:02:18.742774504 (3h23m41.972259851s) [upload=3517]: Uploading segments... Aug 27 21:02:18.742916050 (3h23m41.972401407s) [upload=3517]: Got next segment. Inline: false Aug 27 21:02:18.742942710 (3h23m41.972428057s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:18.742946277 (3h23m41.972431634s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:22.763776455 (3h23m45.993261832s) [seg_pos={0 0}, upload=3517]: Begin upload segment... Aug 27 21:02:22.763826518 (3h23m45.993311906s) [seg_pos={0 0}, upload=3517]: Done begin upload segment. Aug 27 21:02:22.763856284 (3h23m45.993341662s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.763877575 (3h23m45.993362953s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764397404 (3h23m45.99388278s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764422130 (3h23m45.993907507s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764862849 (3h23m45.994348237s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764927922 (3h23m45.994413298s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765295312 (3h23m45.99478069s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765414958 (3h23m45.994900335s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765749918 (3h23m45.995235295s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766207238 (3h23m45.995692616s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766610447 (3h23m45.996095824s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766966717 (3h23m45.996452094s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767231696 (3h23m45.996717073s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767550926 (3h23m45.997036283s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767825002 (3h23m45.99731036s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.768089159 (3h23m45.997574507s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.887072978 (3h23m46.116558345s) [upload=3517]: Next returned nil segment Aug 27 21:02:22.887088627 (3h23m46.116573994s) [upload=3517]: Waiting for error group managing segments... Aug 27 21:02:22.896000215 (3h23m46.125485581s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.897639208 (3h23m46.127124576s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.898268773 (3h23m46.12775413s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.899821925 (3h23m46.129307282s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900364926 (3h23m46.129850283s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900542851 (3h23m46.130028228s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.901166505 (3h23m46.130651862s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.902796171 (3h23m46.132281538s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.904269574 (3h23m46.13375491s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.905707979 (3h23m46.135193326s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.907552551 (3h23m46.137037897s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.908524448 (3h23m46.138009795s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914085382 (3h23m46.143570729s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914102324 (3h23m46.143587671s) [seg_pos={0 0}, upload=3517]: Segment reached optimal threshold of 13 pieces. Aug 27 21:02:22.914233530 (3h23m46.143718877s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914300046 (3h23m46.143785392s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914345892 (3h23m46.143831249s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914401045 (3h23m46.143886412s) [seg_pos={0 0}, upload=3517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 27 21:02:22.914505472 (3h23m46.143990829s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:22.914511263 (3h23m46.143996629s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:24.939283761 (3h23m48.168769138s) [upload=3517]: Done uploading segments... Aug 27 21:02:24.939309329 (3h23m48.168794706s) [upload=3517]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled Duration: 443.077042ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 375.358584ms FAIL (2025-08-27 21:24:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 375.387849ms core uplink us1
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 39.187987ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:24:22 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 39.207132ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 148.983089ms FAIL (2025-08-27 21:24:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 149.007965ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X.............X..................................] duration: 468.949307ms FAIL (2025-08-27 21:24:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 468.971929ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 46.42928ms remote: "tcp:34.120.119.150:443" response headers: Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:14:56 GMT Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 target: https://www.tardigrade.io/ FAIL (2025-08-27 21:24:09 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 46.447706ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 219.717638ms FAIL (2025-08-27 21:24:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 219.738437ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 166.53926ms FAIL (2025-08-27 21:23:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 166.579084ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 161.396521ms FAIL (2025-08-27 21:23:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 27 21:01:41.377970271 (3h23m4.607455648s) [upload=3507]: Starting upload Aug 27 21:01:41.378099755 (3h23m4.607585132s) [upload=3507]: Done starting upload Aug 27 21:01:41.378244187 (3h23m4.607729564s) [upload=3507]: Uploading segments... Aug 27 21:01:41.378310020 (3h23m4.607795397s) [upload=3507]: Got next segment. Inline: true Aug 27 21:01:41.378315851 (3h23m4.607801238s) [upload=3507]: Waiting for error group managing segments... Aug 27 21:01:41.378361376 (3h23m4.607846764s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:41.378366396 (3h23m4.607851773s) [upload=3507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:01:41.378370033 (3h23m4.607855409s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:01:53.431192035 (3h23m16.660677392s) [upload=3507]: Done uploading segments... Aug 27 21:01:53.431206482 (3h23m16.660691839s) [upload=3507]: Upload finished. err: <nil> Duration:* 161.424745ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 702.511069ms FAIL (2025-08-27 21:23:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Aug 27 21:01:36.226560503 (3h22m59.456045849s) [upload=3504]: Starting upload Aug 27 21:01:36.226622129 (3h22m59.456107476s) [upload=3504]: Done starting upload Aug 27 21:01:36.226666592 (3h22m59.456151949s) [upload=3504]: Uploading segments... Aug 27 21:01:36.226834699 (3h22m59.456320045s) [upload=3504]: Got next segment. Inline: false Aug 27 21:01:36.226865156 (3h22m59.456350502s) [upload=3504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:36.226869904 (3h22m59.456355252s) [upload=3504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:36.212957093 (3h23m59.44244245s) [upload=3504]: Next segment err: context deadline exceeded Aug 27 21:02:36.213033316 (3h23m59.442518673s) [upload=3504]: Done uploading segments... Aug 27 21:02:36.213048534 (3h23m59.442533891s) [upload=3504]: Upload finished. err: context deadline exceeded Duration: 702.556304ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 393.426807ms FAIL (2025-08-27 21:23:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:18.742567354 (3h23m41.972052711s) [upload=3517]: Starting upload Aug 27 21:02:18.742644770 (3h23m41.972130117s) [upload=3517]: Done starting upload Aug 27 21:02:18.742774504 (3h23m41.972259851s) [upload=3517]: Uploading segments... Aug 27 21:02:18.742916050 (3h23m41.972401407s) [upload=3517]: Got next segment. Inline: false Aug 27 21:02:18.742942710 (3h23m41.972428057s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:18.742946277 (3h23m41.972431634s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:22.763776455 (3h23m45.993261832s) [seg_pos={0 0}, upload=3517]: Begin upload segment... Aug 27 21:02:22.763826518 (3h23m45.993311906s) [seg_pos={0 0}, upload=3517]: Done begin upload segment. Aug 27 21:02:22.763856284 (3h23m45.993341662s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.763877575 (3h23m45.993362953s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764397404 (3h23m45.99388278s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764422130 (3h23m45.993907507s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764862849 (3h23m45.994348237s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764927922 (3h23m45.994413298s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765295312 (3h23m45.99478069s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765414958 (3h23m45.994900335s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765749918 (3h23m45.995235295s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766207238 (3h23m45.995692616s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766610447 (3h23m45.996095824s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766966717 (3h23m45.996452094s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767231696 (3h23m45.996717073s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767550926 (3h23m45.997036283s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767825002 (3h23m45.99731036s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.768089159 (3h23m45.997574507s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.887072978 (3h23m46.116558345s) [upload=3517]: Next returned nil segment Aug 27 21:02:22.887088627 (3h23m46.116573994s) [upload=3517]: Waiting for error group managing segments... Aug 27 21:02:22.896000215 (3h23m46.125485581s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.897639208 (3h23m46.127124576s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.898268773 (3h23m46.12775413s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.899821925 (3h23m46.129307282s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900364926 (3h23m46.129850283s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900542851 (3h23m46.130028228s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.901166505 (3h23m46.130651862s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.902796171 (3h23m46.132281538s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.904269574 (3h23m46.13375491s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.905707979 (3h23m46.135193326s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.907552551 (3h23m46.137037897s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.908524448 (3h23m46.138009795s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914085382 (3h23m46.143570729s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914102324 (3h23m46.143587671s) [seg_pos={0 0}, upload=3517]: Segment reached optimal threshold of 13 pieces. Aug 27 21:02:22.914233530 (3h23m46.143718877s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914300046 (3h23m46.143785392s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914345892 (3h23m46.143831249s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914401045 (3h23m46.143886412s) [seg_pos={0 0}, upload=3517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 27 21:02:22.914505472 (3h23m46.143990829s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:22.914511263 (3h23m46.143996629s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:24.939283761 (3h23m48.168769138s) [upload=3517]: Done uploading segments... Aug 27 21:02:24.939309329 (3h23m48.168794706s) [upload=3517]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled Duration: 393.47128ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.712572352s FAIL (2025-08-27 21:23:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.712589656s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X.............X.................................] duration: 409.134309ms FAIL (2025-08-27 21:23:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 409.160579ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 50.830324ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:23:32 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 50.936422ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.027870417s FAIL (2025-08-27 21:23:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Aug 27 21:01:36.226560503 (3h22m59.456045849s) [upload=3504]: Starting upload Aug 27 21:01:36.226622129 (3h22m59.456107476s) [upload=3504]: Done starting upload Aug 27 21:01:36.226666592 (3h22m59.456151949s) [upload=3504]: Uploading segments... Aug 27 21:01:36.226834699 (3h22m59.456320045s) [upload=3504]: Got next segment. Inline: false Aug 27 21:01:36.226865156 (3h22m59.456350502s) [upload=3504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:36.226869904 (3h22m59.456355252s) [upload=3504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:36.212957093 (3h23m59.44244245s) [upload=3504]: Next segment err: context deadline exceeded Aug 27 21:02:36.213033316 (3h23m59.442518673s) [upload=3504]: Done uploading segments... Aug 27 21:02:36.213048534 (3h23m59.442533891s) [upload=3504]: Upload finished. err: context deadline exceeded Duration: 1.027884363s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 493.955984ms FAIL (2025-08-27 21:23:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:18.742567354 (3h23m41.972052711s) [upload=3517]: Starting upload Aug 27 21:02:18.742644770 (3h23m41.972130117s) [upload=3517]: Done starting upload Aug 27 21:02:18.742774504 (3h23m41.972259851s) [upload=3517]: Uploading segments... Aug 27 21:02:18.742916050 (3h23m41.972401407s) [upload=3517]: Got next segment. Inline: false Aug 27 21:02:18.742942710 (3h23m41.972428057s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:18.742946277 (3h23m41.972431634s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:22.763776455 (3h23m45.993261832s) [seg_pos={0 0}, upload=3517]: Begin upload segment... Aug 27 21:02:22.763826518 (3h23m45.993311906s) [seg_pos={0 0}, upload=3517]: Done begin upload segment. Aug 27 21:02:22.763856284 (3h23m45.993341662s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.763877575 (3h23m45.993362953s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764397404 (3h23m45.99388278s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764422130 (3h23m45.993907507s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764862849 (3h23m45.994348237s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764927922 (3h23m45.994413298s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765295312 (3h23m45.99478069s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765414958 (3h23m45.994900335s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765749918 (3h23m45.995235295s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766207238 (3h23m45.995692616s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766610447 (3h23m45.996095824s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766966717 (3h23m45.996452094s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767231696 (3h23m45.996717073s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767550926 (3h23m45.997036283s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767825002 (3h23m45.99731036s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.768089159 (3h23m45.997574507s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.887072978 (3h23m46.116558345s) [upload=3517]: Next returned nil segment Aug 27 21:02:22.887088627 (3h23m46.116573994s) [upload=3517]: Waiting for error group managing segments... Aug 27 21:02:22.896000215 (3h23m46.125485581s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.897639208 (3h23m46.127124576s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.898268773 (3h23m46.12775413s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.899821925 (3h23m46.129307282s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900364926 (3h23m46.129850283s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900542851 (3h23m46.130028228s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.901166505 (3h23m46.130651862s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.902796171 (3h23m46.132281538s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.904269574 (3h23m46.13375491s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.905707979 (3h23m46.135193326s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.907552551 (3h23m46.137037897s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.908524448 (3h23m46.138009795s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914085382 (3h23m46.143570729s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914102324 (3h23m46.143587671s) [seg_pos={0 0}, upload=3517]: Segment reached optimal threshold of 13 pieces. Aug 27 21:02:22.914233530 (3h23m46.143718877s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914300046 (3h23m46.143785392s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914345892 (3h23m46.143831249s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914401045 (3h23m46.143886412s) [seg_pos={0 0}, upload=3517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 27 21:02:22.914505472 (3h23m46.143990829s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:22.914511263 (3h23m46.143996629s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:24.939283761 (3h23m48.168769138s) [upload=3517]: Done uploading segments... Aug 27 21:02:24.939309329 (3h23m48.168794706s) [upload=3517]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled Duration: 493.989078ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X.............X................................] duration: 465.809798ms FAIL (2025-08-27 21:23:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 465.838904ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 219.649782ms FAIL (2025-08-27 21:23:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 27 21:01:41.377970271 (3h23m4.607455648s) [upload=3507]: Starting upload Aug 27 21:01:41.378099755 (3h23m4.607585132s) [upload=3507]: Done starting upload Aug 27 21:01:41.378244187 (3h23m4.607729564s) [upload=3507]: Uploading segments... Aug 27 21:01:41.378310020 (3h23m4.607795397s) [upload=3507]: Got next segment. Inline: true Aug 27 21:01:41.378315851 (3h23m4.607801238s) [upload=3507]: Waiting for error group managing segments... Aug 27 21:01:41.378361376 (3h23m4.607846764s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:41.378366396 (3h23m4.607851773s) [upload=3507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:01:41.378370033 (3h23m4.607855409s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:01:53.431192035 (3h23m16.660677392s) [upload=3507]: Done uploading segments... Aug 27 21:01:53.431206482 (3h23m16.660691839s) [upload=3507]: Upload finished. err: <nil> Duration:* 219.670721ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 168.639851ms FAIL (2025-08-27 21:23:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 168.667182ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X.............X...............................] duration: 833.982044ms FAIL (2025-08-27 21:22:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 833.999838ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 285.37078ms FAIL (2025-08-27 21:22:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 285.389685ms core uplink us1
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 44.729913ms remote: "tcp:34.120.119.150:443" response headers: Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:14:56 GMT Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 target: https://www.tardigrade.io/ FAIL (2025-08-27 21:22:58 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 44.756865ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 192.583303ms FAIL (2025-08-27 21:22:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 27 21:01:41.377970271 (3h23m4.607455648s) [upload=3507]: Starting upload Aug 27 21:01:41.378099755 (3h23m4.607585132s) [upload=3507]: Done starting upload Aug 27 21:01:41.378244187 (3h23m4.607729564s) [upload=3507]: Uploading segments... Aug 27 21:01:41.378310020 (3h23m4.607795397s) [upload=3507]: Got next segment. Inline: true Aug 27 21:01:41.378315851 (3h23m4.607801238s) [upload=3507]: Waiting for error group managing segments... Aug 27 21:01:41.378361376 (3h23m4.607846764s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:41.378366396 (3h23m4.607851773s) [upload=3507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:01:41.378370033 (3h23m4.607855409s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:01:53.431192035 (3h23m16.660677392s) [upload=3507]: Done uploading segments... Aug 27 21:01:53.431206482 (3h23m16.660691839s) [upload=3507]: Upload finished. err: <nil> Duration:* 192.612748ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 195.396723ms FAIL (2025-08-27 21:22:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 195.418032ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 467.151005ms FAIL (2025-08-27 21:22:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Aug 27 21:01:36.226560503 (3h22m59.456045849s) [upload=3504]: Starting upload Aug 27 21:01:36.226622129 (3h22m59.456107476s) [upload=3504]: Done starting upload Aug 27 21:01:36.226666592 (3h22m59.456151949s) [upload=3504]: Uploading segments... Aug 27 21:01:36.226834699 (3h22m59.456320045s) [upload=3504]: Got next segment. Inline: false Aug 27 21:01:36.226865156 (3h22m59.456350502s) [upload=3504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:36.226869904 (3h22m59.456355252s) [upload=3504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:36.212957093 (3h23m59.44244245s) [upload=3504]: Next segment err: context deadline exceeded Aug 27 21:02:36.213033316 (3h23m59.442518673s) [upload=3504]: Done uploading segments... Aug 27 21:02:36.213048534 (3h23m59.442533891s) [upload=3504]: Upload finished. err: context deadline exceeded Duration: 467.178055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X.............X..............................] duration: 558.673976ms FAIL (2025-08-27 21:22:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 558.698732ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.791198291s FAIL (2025-08-27 21:22:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.791224421s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 390.171792ms FAIL (2025-08-27 21:22:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:18.742567354 (3h23m41.972052711s) [upload=3517]: Starting upload Aug 27 21:02:18.742644770 (3h23m41.972130117s) [upload=3517]: Done starting upload Aug 27 21:02:18.742774504 (3h23m41.972259851s) [upload=3517]: Uploading segments... Aug 27 21:02:18.742916050 (3h23m41.972401407s) [upload=3517]: Got next segment. Inline: false Aug 27 21:02:18.742942710 (3h23m41.972428057s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:18.742946277 (3h23m41.972431634s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:22.763776455 (3h23m45.993261832s) [seg_pos={0 0}, upload=3517]: Begin upload segment... Aug 27 21:02:22.763826518 (3h23m45.993311906s) [seg_pos={0 0}, upload=3517]: Done begin upload segment. Aug 27 21:02:22.763856284 (3h23m45.993341662s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.763877575 (3h23m45.993362953s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764397404 (3h23m45.99388278s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764422130 (3h23m45.993907507s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764862849 (3h23m45.994348237s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764927922 (3h23m45.994413298s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765295312 (3h23m45.99478069s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765414958 (3h23m45.994900335s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765749918 (3h23m45.995235295s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766207238 (3h23m45.995692616s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766610447 (3h23m45.996095824s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766966717 (3h23m45.996452094s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767231696 (3h23m45.996717073s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767550926 (3h23m45.997036283s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767825002 (3h23m45.99731036s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.768089159 (3h23m45.997574507s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.887072978 (3h23m46.116558345s) [upload=3517]: Next returned nil segment Aug 27 21:02:22.887088627 (3h23m46.116573994s) [upload=3517]: Waiting for error group managing segments... Aug 27 21:02:22.896000215 (3h23m46.125485581s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.897639208 (3h23m46.127124576s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.898268773 (3h23m46.12775413s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.899821925 (3h23m46.129307282s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900364926 (3h23m46.129850283s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900542851 (3h23m46.130028228s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.901166505 (3h23m46.130651862s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.902796171 (3h23m46.132281538s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.904269574 (3h23m46.13375491s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.905707979 (3h23m46.135193326s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.907552551 (3h23m46.137037897s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.908524448 (3h23m46.138009795s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914085382 (3h23m46.143570729s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914102324 (3h23m46.143587671s) [seg_pos={0 0}, upload=3517]: Segment reached optimal threshold of 13 pieces. Aug 27 21:02:22.914233530 (3h23m46.143718877s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914300046 (3h23m46.143785392s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914345892 (3h23m46.143831249s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914401045 (3h23m46.143886412s) [seg_pos={0 0}, upload=3517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 27 21:02:22.914505472 (3h23m46.143990829s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:22.914511263 (3h23m46.143996629s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:24.939283761 (3h23m48.168769138s) [upload=3517]: Done uploading segments... Aug 27 21:02:24.939309329 (3h23m48.168794706s) [upload=3517]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled Duration: 390.216156ms core uplink us-select-1
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 43.613307ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:22:20 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 43.63147ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 540.608738ms FAIL (2025-08-27 21:22:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Aug 27 21:01:36.226560503 (3h22m59.456045849s) [upload=3504]: Starting upload Aug 27 21:01:36.226622129 (3h22m59.456107476s) [upload=3504]: Done starting upload Aug 27 21:01:36.226666592 (3h22m59.456151949s) [upload=3504]: Uploading segments... Aug 27 21:01:36.226834699 (3h22m59.456320045s) [upload=3504]: Got next segment. Inline: false Aug 27 21:01:36.226865156 (3h22m59.456350502s) [upload=3504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:36.226869904 (3h22m59.456355252s) [upload=3504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:36.212957093 (3h23m59.44244245s) [upload=3504]: Next segment err: context deadline exceeded Aug 27 21:02:36.213033316 (3h23m59.442518673s) [upload=3504]: Done uploading segments... Aug 27 21:02:36.213048534 (3h23m59.442533891s) [upload=3504]: Upload finished. err: context deadline exceeded Duration: 540.637601ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X.............X.............................] duration: 764.921172ms FAIL (2025-08-27 21:22:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 764.948294ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 306.321616ms FAIL (2025-08-27 21:22:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 27 21:01:41.377970271 (3h23m4.607455648s) [upload=3507]: Starting upload Aug 27 21:01:41.378099755 (3h23m4.607585132s) [upload=3507]: Done starting upload Aug 27 21:01:41.378244187 (3h23m4.607729564s) [upload=3507]: Uploading segments... Aug 27 21:01:41.378310020 (3h23m4.607795397s) [upload=3507]: Got next segment. Inline: true Aug 27 21:01:41.378315851 (3h23m4.607801238s) [upload=3507]: Waiting for error group managing segments... Aug 27 21:01:41.378361376 (3h23m4.607846764s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:41.378366396 (3h23m4.607851773s) [upload=3507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:01:41.378370033 (3h23m4.607855409s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:01:53.431192035 (3h23m16.660677392s) [upload=3507]: Done uploading segments... Aug 27 21:01:53.431206482 (3h23m16.660691839s) [upload=3507]: Upload finished. err: <nil> Duration:* 306.349729ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 267.783691ms FAIL (2025-08-27 21:22:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 267.807746ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 436.473707ms FAIL (2025-08-27 21:22:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 436.514363ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.619442791s FAIL (2025-08-27 21:21:46 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.619547168s core uplink us1
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 43.517806ms remote: "tcp:34.120.119.150:443" response headers: Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:14:56 GMT Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 target: https://www.tardigrade.io/ FAIL (2025-08-27 21:21:45 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 43.588939ms core website
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 50.137518ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:21:34 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 50.156404ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 188.950373ms FAIL (2025-08-27 21:21:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 188.974457ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 160.124009ms FAIL (2025-08-27 21:21:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 160.143144ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 405.041508ms FAIL (2025-08-27 21:21:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:18.742567354 (3h23m41.972052711s) [upload=3517]: Starting upload Aug 27 21:02:18.742644770 (3h23m41.972130117s) [upload=3517]: Done starting upload Aug 27 21:02:18.742774504 (3h23m41.972259851s) [upload=3517]: Uploading segments... Aug 27 21:02:18.742916050 (3h23m41.972401407s) [upload=3517]: Got next segment. Inline: false Aug 27 21:02:18.742942710 (3h23m41.972428057s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:18.742946277 (3h23m41.972431634s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:22.763776455 (3h23m45.993261832s) [seg_pos={0 0}, upload=3517]: Begin upload segment... Aug 27 21:02:22.763826518 (3h23m45.993311906s) [seg_pos={0 0}, upload=3517]: Done begin upload segment. Aug 27 21:02:22.763856284 (3h23m45.993341662s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.763877575 (3h23m45.993362953s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764397404 (3h23m45.99388278s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764422130 (3h23m45.993907507s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764862849 (3h23m45.994348237s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764927922 (3h23m45.994413298s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765295312 (3h23m45.99478069s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765414958 (3h23m45.994900335s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765749918 (3h23m45.995235295s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766207238 (3h23m45.995692616s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766610447 (3h23m45.996095824s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766966717 (3h23m45.996452094s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767231696 (3h23m45.996717073s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767550926 (3h23m45.997036283s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767825002 (3h23m45.99731036s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.768089159 (3h23m45.997574507s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.887072978 (3h23m46.116558345s) [upload=3517]: Next returned nil segment Aug 27 21:02:22.887088627 (3h23m46.116573994s) [upload=3517]: Waiting for error group managing segments... Aug 27 21:02:22.896000215 (3h23m46.125485581s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.897639208 (3h23m46.127124576s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.898268773 (3h23m46.12775413s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.899821925 (3h23m46.129307282s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900364926 (3h23m46.129850283s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900542851 (3h23m46.130028228s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.901166505 (3h23m46.130651862s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.902796171 (3h23m46.132281538s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.904269574 (3h23m46.13375491s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.905707979 (3h23m46.135193326s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.907552551 (3h23m46.137037897s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.908524448 (3h23m46.138009795s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914085382 (3h23m46.143570729s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914102324 (3h23m46.143587671s) [seg_pos={0 0}, upload=3517]: Segment reached optimal threshold of 13 pieces. Aug 27 21:02:22.914233530 (3h23m46.143718877s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914300046 (3h23m46.143785392s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914345892 (3h23m46.143831249s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914401045 (3h23m46.143886412s) [seg_pos={0 0}, upload=3517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 27 21:02:22.914505472 (3h23m46.143990829s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:22.914511263 (3h23m46.143996629s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:24.939283761 (3h23m48.168769138s) [upload=3517]: Done uploading segments... Aug 27 21:02:24.939309329 (3h23m48.168794706s) [upload=3517]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled Duration: 405.056506ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X.............X............................] duration: 660.471878ms FAIL (2025-08-27 21:21:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8a256c146cd8f37955e142ac84d0faa38bd579ebb43ee626e10b5a173232e949 != before 4180454ac5ddc2c2ed56c97427218ce6f6f5d85b0003f9cbb032a89f4ea995f0 ======LOGS====== Aug 27 21:01:32.750878783 (3h22m55.98036416s) [upload=3502]: Starting upload Aug 27 21:01:32.750950248 (3h22m55.980435625s) [upload=3502]: Done starting upload Aug 27 21:01:32.751067818 (3h22m55.980553185s) [upload=3502]: Uploading segments... Aug 27 21:01:32.751334860 (3h22m55.980820237s) [upload=3502]: Got next segment. Inline: false Aug 27 21:01:32.751395275 (3h22m55.980880651s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:32.751400624 (3h22m55.980886012s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:21.290948080 (3h23m44.520433427s) [seg_pos={0 0}, upload=3502]: Begin upload segment... Aug 27 21:02:21.290984167 (3h23m44.520469524s) [seg_pos={0 0}, upload=3502]: Done begin upload segment. Aug 27 21:02:21.291006769 (3h23m44.520492127s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291142315 (3h23m44.520627692s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291344345 (3h23m44.520829702s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291696138 (3h23m44.521181515s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.291900602 (3h23m44.52138598s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292130163 (3h23m44.521615541s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292559302 (3h23m44.522044679s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292575773 (3h23m44.522061149s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.292633481 (3h23m44.522118858s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293075643 (3h23m44.52256101s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293144252 (3h23m44.522629628s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293448895 (3h23m44.522934272s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293757236 (3h23m44.523242622s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.293842716 (3h23m44.523328093s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294236097 (3h23m44.523721473s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294313281 (3h23m44.523798668s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.294949088 (3h23m44.524434464s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.295707735 (3h23m44.525193113s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Uploading piece... Aug 27 21:02:21.424581752 (3h23m44.654067109s) [upload=3502]: Next returned nil segment Aug 27 21:02:21.424595878 (3h23m44.654081245s) [upload=3502]: Waiting for error group managing segments... Aug 27 21:02:21.433819111 (3h23m44.663304479s) [noise=true , address=79.127.219.47:4056 , piece_id=RRCIXM76CJXQ57WYWAUXA6ACD64DJMCEKHKTTHPETPDLCGCWUIRA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.435597317 (3h23m44.665082684s) [noise=true , address=79.127.219.51:4197 , piece_id=56VI4UYTMCHF72PWZNFD2HTSX335N27CGVRKG7WC5JHD4BDGHG4Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.440452071 (3h23m44.669937439s) [noise=true , address=104.36.82.9:4038 , piece_id=CSPYJUZ5VGFK5DSIKIY5Q6DRLJJE4RJI4ZQ7OCKGELIUME7W3QVA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.443653585 (3h23m44.673138942s) [noise=true , address=104.36.82.41:4788 , piece_id=S7TS2ZIIVJG46WDR6JOB4DENJ7NW4WCZXW2UCXTNZKBTRHXRT2OQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.446290125 (3h23m44.675775473s) [noise=true , address=45.91.116.11:4124 , piece_id=Y4K6VNMU5DIEEUB2CIUJNZY63HLRYS2JZJXUTMTVA4NLNACCHWDQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.453133401 (3h23m44.682618768s) [noise=true , address=104.36.82.52:4088 , piece_id=4WJLZXA5ZC2XOKWC5XLIGWLBS66UPEXULKFMFY3RE47U6FVGQUGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.456748804 (3h23m44.686234181s) [noise=true , address=77.90.31.21:4897 , piece_id=EMB3LDMFCWRW4BKPNARAWGUYU5LFZ3KTLR7HS6F66JUL7T4FMLGQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.458665269 (3h23m44.688150636s) [noise=true , address=77.90.31.16:4738 , piece_id=V4GJFJMD6H55IJX4F3M7R27SCJCJ2EIV5JU6L6EKUKENQPT77DMA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.462817522 (3h23m44.692302889s) [noise=true , address=77.90.31.15:4708 , piece_id=D3PZXJEURKU57E5K5MECL3DOM2B42M5QUOVFJFD36WRT6UYCXT3Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.465059280 (3h23m44.694544657s) [noise=true , address=104.36.82.61:4378 , piece_id=YSROKHT2OELO3P57EIKKU4HVFL7PUMLZYZPQYRF75M3Q66OUL5DQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.467399463 (3h23m44.69688484s) [noise=true , address=104.36.82.16:4315 , piece_id=TAVXG3NTXLYYR4ZAZE3O476ZKGGXVMSJ2J32UGMCJ5XCR2RV5WQA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.471858793 (3h23m44.701344171s) [noise=true , address=104.36.82.37:4687 , piece_id=H2CXH33EV4SV7RB7B6WOOLH72KZKIE36PD4RL5F6XBCXTQ26NGVQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.480471758 (3h23m44.709957145s) [noise=true , address=104.36.82.11:4125 , piece_id=T2L6SJ3AIZYWDLJHMCKFQNFVE2MX2RRCF7WOGQBLFFDAV4FTIRUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568716922 (3h23m44.798202269s) [noise=true , address=77.90.31.19:4853 , piece_id=2UQSONX56FRWWV3JCIMOB7CU3WFUX54EHGZ5442RFF3UU3SKQGYA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: <nil> Aug 27 21:02:21.568741268 (3h23m44.798226625s) [seg_pos={0 0}, upload=3502]: Segment reached optimal threshold of 14 pieces. Aug 27 21:02:21.568847017 (3h23m44.798332374s) [noise=true , address=79.127.201.215:4358 , piece_id=AUKEJZCQO4S5UOSQ3QLLCBSWNZLF7B4SOINJW4OMY5MPHIYLVW2Q, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568907941 (3h23m44.798393288s) [noise=true , address=77.90.31.10:4521 , piece_id=SVHWKNNKEVII3E3ABZGFDR2EOS7WRS6ATG3LKARMGNLGQZXWBTNQ, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WpVJa7SXkANKParDWYrC1cN8aLHTkUSPrRJGKQ6fvA5oDEkoa): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568944320 (3h23m44.798429677s) [noise=true , address=79.127.205.245:4221 , piece_id=BGIYKF74JI7SDRQRJSPFVLW2HSTS7THZCO5372JY7XYY2TC4VXUA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mRTHdVPMGp5pANgUX5B657RvXmBo9fD1JfLJHNsYGRnPgczZ4): protocol: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:48984->79.127.205.245:4221: use of closed network connection Aug 27 21:02:21.568961932 (3h23m44.798447289s) [noise=true , address=77.90.31.17:4781 , piece_id=RM72FQ2B5N4AMUSHD5BL7CDSIUQ2FVRYMIXIQH3UGYSJMW2XZXDA, seg_pos={0 0}, upload=3502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Uc2Hn41mUYgLtyirRtEGNv5iKBv8YqzQo3RYdoE4M1Fj29WgT): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:21.568983242 (3h23m44.7984686s) [seg_pos={0 0}, upload=3502]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 27 21:02:21.569081298 (3h23m44.798566655s) [upload=3502]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:21.569085866 (3h23m44.798571223s) [upload=3502]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:29.941659943 (3h23m53.1711453s) [upload=3502]: Done uploading segments... Aug 27 21:02:29.941674550 (3h23m53.171159907s) [upload=3502]: Upload finished. err: <nil> Duration: 660.517625ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 156.010163ms FAIL (2025-08-27 21:21:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 27 21:01:41.377970271 (3h23m4.607455648s) [upload=3507]: Starting upload Aug 27 21:01:41.378099755 (3h23m4.607585132s) [upload=3507]: Done starting upload Aug 27 21:01:41.378244187 (3h23m4.607729564s) [upload=3507]: Uploading segments... Aug 27 21:01:41.378310020 (3h23m4.607795397s) [upload=3507]: Got next segment. Inline: true Aug 27 21:01:41.378315851 (3h23m4.607801238s) [upload=3507]: Waiting for error group managing segments... Aug 27 21:01:41.378361376 (3h23m4.607846764s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:41.378366396 (3h23m4.607851773s) [upload=3507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:01:41.378370033 (3h23m4.607855409s) [upload=3507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:01:53.431192035 (3h23m16.660677392s) [upload=3507]: Done uploading segments... Aug 27 21:01:53.431206482 (3h23m16.660691839s) [upload=3507]: Upload finished. err: <nil> Duration:* 156.031976ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 574.58911ms FAIL (2025-08-27 21:21:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Aug 27 21:01:36.226560503 (3h22m59.456045849s) [upload=3504]: Starting upload Aug 27 21:01:36.226622129 (3h22m59.456107476s) [upload=3504]: Done starting upload Aug 27 21:01:36.226666592 (3h22m59.456151949s) [upload=3504]: Uploading segments... Aug 27 21:01:36.226834699 (3h22m59.456320045s) [upload=3504]: Got next segment. Inline: false Aug 27 21:01:36.226865156 (3h22m59.456350502s) [upload=3504]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:01:36.226869904 (3h22m59.456355252s) [upload=3504]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:36.212957093 (3h23m59.44244245s) [upload=3504]: Next segment err: context deadline exceeded Aug 27 21:02:36.213033316 (3h23m59.442518673s) [upload=3504]: Done uploading segments... Aug 27 21:02:36.213048534 (3h23m59.442533891s) [upload=3504]: Upload finished. err: context deadline exceeded Duration: 574.626942ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.84357977s FAIL (2025-08-27 21:21:01 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.843598605s core uplink us1
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 42.369313ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:21:01 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 42.387978ms core website
checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 40.204391ms remote: "tcp:34.120.119.150:443" response headers: Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:14:56 GMT Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 target: https://www.tardigrade.io/ FAIL (2025-08-27 21:20:51 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 40.224719ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 423.624693ms FAIL (2025-08-27 21:20:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:07.426688152 (3h23m30.656173499s) [upload=3515]: Starting upload Aug 27 21:02:07.426735873 (3h23m30.65622122s) [upload=3515]: Done starting upload Aug 27 21:02:07.426790635 (3h23m30.656275992s) [upload=3515]: Uploading segments... Aug 27 21:02:07.426837303 (3h23m30.65632265s) [upload=3515]: Got next segment. Inline: true Aug 27 21:02:07.426841050 (3h23m30.656326387s) [upload=3515]: Waiting for error group managing segments... Aug 27 21:02:07.426872439 (3h23m30.656357786s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:07.426875605 (3h23m30.656360952s) [upload=3515]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 27 21:02:07.426877649 (3h23m30.656362995s) [upload=3515]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:20.276735982 (3h23m43.506221359s) [upload=3515]: Done uploading segments... Aug 27 21:02:20.276756050 (3h23m43.506241427s) [upload=3515]: Upload finished. err: <nil> Duration:* 423.64904ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 602.580631ms FAIL (2025-08-27 21:20:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled ======LOGS====== Aug 27 21:02:18.742567354 (3h23m41.972052711s) [upload=3517]: Starting upload Aug 27 21:02:18.742644770 (3h23m41.972130117s) [upload=3517]: Done starting upload Aug 27 21:02:18.742774504 (3h23m41.972259851s) [upload=3517]: Uploading segments... Aug 27 21:02:18.742916050 (3h23m41.972401407s) [upload=3517]: Got next segment. Inline: false Aug 27 21:02:18.742942710 (3h23m41.972428057s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 27 21:02:18.742946277 (3h23m41.972431634s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 27 21:02:22.763776455 (3h23m45.993261832s) [seg_pos={0 0}, upload=3517]: Begin upload segment... Aug 27 21:02:22.763826518 (3h23m45.993311906s) [seg_pos={0 0}, upload=3517]: Done begin upload segment. Aug 27 21:02:22.763856284 (3h23m45.993341662s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.763877575 (3h23m45.993362953s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764397404 (3h23m45.99388278s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764422130 (3h23m45.993907507s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764862849 (3h23m45.994348237s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.764927922 (3h23m45.994413298s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765295312 (3h23m45.99478069s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765414958 (3h23m45.994900335s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.765749918 (3h23m45.995235295s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766207238 (3h23m45.995692616s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766610447 (3h23m45.996095824s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.766966717 (3h23m45.996452094s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767231696 (3h23m45.996717073s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767550926 (3h23m45.997036283s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.767825002 (3h23m45.99731036s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.768089159 (3h23m45.997574507s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Uploading piece... Aug 27 21:02:22.887072978 (3h23m46.116558345s) [upload=3517]: Next returned nil segment Aug 27 21:02:22.887088627 (3h23m46.116573994s) [upload=3517]: Waiting for error group managing segments... Aug 27 21:02:22.896000215 (3h23m46.125485581s) [noise=true , address=45.91.116.6:4974 , piece_id=5QXPGI3XYTQL32VUXYMOQ54XLUJEAHZBIBONW767YOT43RA6PBIQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.897639208 (3h23m46.127124576s) [noise=true , address=45.91.116.12:4227 , piece_id=SQKEVOYWMFBKXCMGFBT7DIYTN7QS4UBRGCFNRLCNWPGHFOJA7XJQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.898268773 (3h23m46.12775413s) [noise=true , address=45.91.116.12:4225 , piece_id=FZGNYKDMAQAYD5XAF2N6SIJMQLA46PVXPN6T35JLSAUQUUCBAYCA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.899821925 (3h23m46.129307282s) [noise=true , address=45.91.116.9:4261 , piece_id=IK5OIGTBU723YGGDN3RPXKUVZ3QC3NEM3T27DYT5LDHW3B4SYPKQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900364926 (3h23m46.129850283s) [noise=true , address=45.91.116.4:4938 , piece_id=SQRBXTCCYOWGL2VAFM75RPTBQFPW6IFY4R5EHYZRF5G6FUPVNMNA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.900542851 (3h23m46.130028228s) [noise=true , address=45.91.116.10:4083 , piece_id=KBS4UACDAU2GLY4JTEDO7SDI2U66Z3EYDIUAZ2IYI5ET5SUTX6CQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.901166505 (3h23m46.130651862s) [noise=true , address=77.90.31.10:4518 , piece_id=46XH35VERNQPW4RJ5ZOHHPH2MU3ADGS6FNOPLOZLPV66MTJDYP5Q, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.902796171 (3h23m46.132281538s) [noise=true , address=104.36.82.9:4005 , piece_id=B5TH2PWMLINVCJI4HVOYWS6B66KIUNPHHY52HWF5O5IB34QFFFPA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.904269574 (3h23m46.13375491s) [noise=true , address=45.91.116.7:4009 , piece_id=UKIZPHON3RLITQ2CWMFWDF7M26EG5DCDDFMV5EQBGK2G6J5ONNMQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.905707979 (3h23m46.135193326s) [noise=true , address=104.36.82.2:4731 , piece_id=MJCWMSTTWQHZOHA5P42TFJDK3QRRFUQ6KN4A6ZBDHMA53AWBRPSA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.907552551 (3h23m46.137037897s) [noise=true , address=104.36.82.15:4259 , piece_id=IDJKXXMXEGK33JGUNUU6XQTDKW2YUL2XTHIVQRLAVMDA7Q6LE75A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.908524448 (3h23m46.138009795s) [noise=true , address=77.90.31.13:4627 , piece_id=LOX4EMR3AU76ALQ7P6RGS6O7ROXVFNQHTPTQTEU2OHU7DKVKMTEA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914085382 (3h23m46.143570729s) [noise=true , address=104.36.82.6:4873 , piece_id=S43FRJTNLACGGL4IREGGZDRDOAL2MBQGCOBYH66WMYQ6QQ6DDPEQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: <nil> Aug 27 21:02:22.914102324 (3h23m46.143587671s) [seg_pos={0 0}, upload=3517]: Segment reached optimal threshold of 13 pieces. Aug 27 21:02:22.914233530 (3h23m46.143718877s) [noise=true , address=104.36.82.16:4294 , piece_id=JJZDI2PNUG5YZ6CSNHGTLYCRN5LW7BWWVH2FRHVH67FWSFVMGDGA, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914300046 (3h23m46.143785392s) [noise=true , address=77.90.31.11:4555 , piece_id=FIRU7P4OW3CSYXWTTTMHYNZFN22R52FJH7I67PVB4FI65FND72VQ, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914345892 (3h23m46.143831249s) [noise=true , address=45.91.116.8:4045 , piece_id=DRKMDKC2XKXARK3XAYY6LP62BRPRB7NDL2B63QS3WKCLK732IX6A, seg_pos={0 0}, upload=3517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 27 21:02:22.914401045 (3h23m46.143886412s) [seg_pos={0 0}, upload=3517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 27 21:02:22.914505472 (3h23m46.143990829s) [upload=3517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 27 21:02:22.914511263 (3h23m46.143996629s) [upload=3517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 27 21:02:24.939283761 (3h23m48.168769138s) [upload=3517]: Done uploading segments... Aug 27 21:02:24.939309329 (3h23m48.168794706s) [upload=3517]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp: lookup redis-master.redis: operation was canceled Duration: 602.611351ms core uplink us-select-1
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 43.335251ms remote: "tcp:34.120.119.150:443" response headers: Alt-Svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 Content-Type=text/html; charset=UTF-8 Referrer-Policy=no-referrer Content-Length=332 Date=Wed, 27 Aug 2025 21:17:16 GMT target: https://storj.io/ FAIL (2025-08-27 21:20:33 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "\n<html><head>\n<meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<title>502 Server Error</title>\n</head>\n<body text=#000000 bgcolor=#ffffff>\n<h1>Error: Server Error</h1>\n<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>\n<h2></h2>\n</body></html>\n" Duration: 43.381991ms core website

Orbiter Check History

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)

checkers.HTTP{"GET" "https://www.tardigrade.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.tardigrade_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)