Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 387.675546ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:38:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 387.699241ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 382.039466ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:38:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 382.056848ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 427.249546ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:38:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 427.272868ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 354.259869ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:38:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 354.281339ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 334.397733ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:37:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 334.417261ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 373.908026ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:37:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 373.927212ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 360.224826ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:37:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 360.236968ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 371.492261ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:36:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 371.515686ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 345.32037ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:36:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 345.332562ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 418.321529ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:36:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 418.347145ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 368.975573ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:36:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 369.004076ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 362.799243ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:36:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 362.960494ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 444.823085ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:36:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 444.872318ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 336.131498ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:35:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 336.150674ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 362.951792ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:35:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 362.973212ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 489.766786ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:34:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 489.779861ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 325.013397ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:34:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 325.030529ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 368.051229ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:34:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 368.064534ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 480.229041ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:33:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 480.253846ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 341.947331ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:33:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 341.962379ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 442.780054ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:33:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 442.799491ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 426.881426ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:33:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 426.899228ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 320.896206ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:32:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 320.916464ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 321.396221ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:32:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 321.40621ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 481.495997ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:32:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 481.515384ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 395.847648ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:32:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 395.925175ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 362.805989ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:31:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 362.828651ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 521.230748ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:31:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 521.24786ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 344.162189ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:31:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 344.188888ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 392.967129ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:31:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 392.985612ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 323.600016ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:30:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 323.619011ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 342.420385ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:30:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 342.440121ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 429.586639ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:30:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 429.607057ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 390.946171ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:30:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 390.970396ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 324.17442ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:29:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 324.195178ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 590.303628ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:29:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 590.32623ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 366.011259ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:29:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 366.029533ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 357.769863ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:29:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 357.791494ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 377.100034ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:29:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 377.115934ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 496.24454ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:29:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 496.270147ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 379.56547ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:28:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 379.582662ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 331.505234ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:28:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 331.523058ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 352.462446ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:28:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 352.480099ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 405.642196ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:28:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 405.666061ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 398.739862ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:27:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 398.756833ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 385.948425ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:27:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 385.975896ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 380.039175ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:27:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 380.060225ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 392.571969ms trace: https://jaeger.dev.storj.io/trace/6a6845937c069bb0 FAIL (2026-02-08 18:27:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 392.594281ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 464.155118ms trace: https://jaeger.dev.storj.io/trace/24ddfb3c4d5076be FAIL (2026-02-08 18:27:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 78h25m17.225417152s) [upload=508534]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:21:39.042976510 (578h25m17.225421481s) [upload=508534]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:21:39.074946726 (578h25m17.257391696s) [seg_pos={0 0}, upload=508534]: Begin upload segment... Feb 8 18:21:39.074985048 (578h25m17.257430018s) [seg_pos={0 0}, upload=508534]: Done begin upload segment. Feb 8 18:21:39.075002911 (578h25m17.257447881s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.077277892 (578h25m17.259722861s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080286433 (578h25m17.262731402s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080542701 (578h25m17.262987671s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.080786447 (578h25m17.263231416s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081094051 (578h25m17.263539021s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081316987 (578h25m17.263761957s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.081577645 (578h25m17.264022615s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.085208628 (578h25m17.267653597s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.086855244 (578h25m17.269300214s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.088638956 (578h25m17.271083925s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.090304568 (578h25m17.272749538s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.091980138 (578h25m17.274425098s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.093715801 (578h25m17.27616077s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.095392643 (578h25m17.277837614s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.097077270 (578h25m17.279522231s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.098136460 (578h25m17.28058142s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.099135626 (578h25m17.281580596s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Uploading piece... Feb 8 18:21:39.150914568 (578h25m17.333359539s) [upload=508534]: Next returned nil segment Feb 8 18:21:39.150928575 (578h25m17.333373535s) [upload=508534]: Waiting for error group managing segments... Feb 8 18:21:39.164678192 (578h25m17.347123162s) [noise=true , address=145.79.227.42:4138 , piece_id=CHA727RTZMCV56TXM3WOL4QDD753X3XUMH5MHF7GAPLMHX2CDNEA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.169414621 (578h25m17.35185958s) [noise=true , address=145.79.227.24:4604 , piece_id=BUB4C4KHOHKMGOANPYSXJDJI4NX4OE6VD5ZI6SPFPGELBPGP37IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.176222318 (578h25m17.358667289s) [noise=true , address=145.79.227.20:4448 , piece_id=RJ2L45IULXHRLGUW3JGXZCBHH2TWT5UERT73GOOZYU6XOACWTYOA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.182362579 (578h25m17.364807559s) [noise=true , address=145.79.227.38:4991 , piece_id=UZKZ2TOYZ4ZY56TLUHOVQQRH4Q6RGQKFPYMKVYMAJ62RK65QGLAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.183073257 (578h25m17.365518217s) [noise=true , address=79.127.219.38:4846 , piece_id=FPQOAFXBTCF7PTJ3FYFEGXCBHX7UVWQVSISS36YKBFDMBG2KZK6Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.195178242 (578h25m17.377623232s) [noise=true , address=145.79.227.33:4884 , piece_id=OIE5AULWGS2RHUN4IZ64OY2S2EUQAKNJYRT43XGGX63NVP7ANWRA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200453396 (578h25m17.382898387s) [noise=true , address=145.79.227.51:4422 , piece_id=TD775B2MXWHVLXTEJY3VSV4OSOYK3FXXXH3RU3ZZIBIJRWRWUTBA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.200802179 (578h25m17.383247148s) [noise=true , address=145.79.227.32:4858 , piece_id=FYQFSQD6DJ74J7EMKIGRNJ3P2D2X2Y4BGRTQRRNJSQCX4CDSKWMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.203036091 (578h25m17.385481062s) [noise=true , address=145.79.227.34:4932 , piece_id=ALG52LB276Y3L25XMTNCW2K73XE45FG5GT2TK77NB3AJNRM2IMYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.223017641 (578h25m17.405462631s) [noise=true , address=79.127.219.41:4932 , piece_id=AFFS7VHDOKEAXDQQNVK3QUW4ZU5FNM5GSDBO6DCW5KT6DMWNMIAQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.249686233 (578h25m17.432131203s) [noise=true , address=79.127.219.45:4876 , piece_id=YVP4ATZTQM5IADFZRXNIMM7ENDJICRK5VTM53CF3R2JXNO6USQ2Q, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:39.659418571 (578h25m17.841863551s) [noise=true , address=64.49.13.170:4722 , piece_id=D3LYC3LSCEGEDWXKDTVNS2IJ6AEJ3AMHTBP75SEWKAHLHSF5PQSQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:21:40.040722118 (578h25m18.223167088s) [noise=true , address=145.79.227.41:4088 , piece_id=OROPXGCQTBMJMNRQ7MOZC5KDMRZHAQ63DYH45BFV3BI5QLWRZSWQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: <nil> Feb 8 18:22:39.028213013 (578h26m17.210657983s) [noise=true , address=77.90.31.10:4549 , piece_id=VNZV4S5SWYFL7K6D74AVGAQG53ELX5NUPILPYAAP5ZXI3LG7P6IQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1Uay43uEvzmHudZaGM1ZcEFbEdUXzQto5RtZNiQXgrqaRmuQoy, address:77.90.31.10:4549): protocol: read tcp 5.161.89.252:43076->77.90.31.10:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028300447 (578h26m17.210745427s) [noise=true , address=77.90.31.15:4714 , piece_id=CNS46RJLATVFJWJBRZP6FB2CKJBZ6ZITE4N5NSWMJBFUNDRO5AYQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: read tcp 5.161.89.252:58064->77.90.31.15:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028320574 (578h26m17.210765543s) [noise=true , address=77.90.31.19:4870 , piece_id=NXQIM3PIVEN6WJZXPZL5ZFRT7M25ZAT5R6DYVHEWIQYA46XEONMQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1hNVXEVe8DW8F8meERGWkxAazEZKLKBeprNn6e5C94eEoAMBaR, address:77.90.31.19:4870): protocol: read tcp 5.161.89.252:61372->77.90.31.19:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028392268 (578h26m17.210837238s) [noise=true , address=77.90.31.13:4660 , piece_id=DM6J3R3BMVE56P3COTTQ5EIGRJBEVLXIVDTNRDTKYQWTZZZDEGPA, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:1TpUo13NwY6M47BnPsnbZD1WRKpX4KRDhYs2m3WH8VBrupnkRG, address:77.90.31.13:4660): protocol: read tcp 5.161.89.252:31294->77.90.31.13:4660: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028447251 (578h26m17.210892221s) [noise=true , address=77.90.31.21:4885 , piece_id=GWI4XWM4OKQC4BWAQEPQR5HVBSJMIMILM423HPDLRGFUMKCAHRJQ, seg_pos={0 0}, upload=508534]: Done uploading piece. err: ecclient: upload failed (node:12TfMGrvCMPvQ9ZC1pVoDqjFnUHXtzwwnZB6pCV1em4pdmbK2MP, address:77.90.31.21:4885): protocol: read tcp 5.161.89.252:23554->77.90.31.21:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:39.028468099 (578h26m17.21091307s) [seg_pos={0 0}, upload=508534]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Feb 8 18:22:39.053128629 (578h26m17.235573609s) [upload=508534]: Done uploading segments... Feb 8 18:22:39.053147775 (578h26m17.235592755s) [upload=508534]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 464.172912ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 328.655222ms trace: https://jaeger.dev.storj.io/trace/385d0c3ed2c22ea6 FAIL (2026-02-08 18:26:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Feb 8 18:22:08.712584950 (578h25m46.89502993s) [upload=508543]: Starting upload Feb 8 18:22:08.712672833 (578h25m46.895117813s) [upload=508543]: Done starting upload Feb 8 18:22:08.712703681 (578h25m46.895148652s) [upload=508543]: Uploading segments... Feb 8 18:22:08.713064396 (578h25m46.895509366s) [upload=508543]: Got next segment. Inline: false Feb 8 18:22:08.713101294 (578h25m46.895546264s) [upload=508543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Feb 8 18:22:08.713106043 (578h25m46.895551013s) [upload=508543]: Flush batch item: pb.BatchRequestItem_SegmentBegin Feb 8 18:22:08.733140912 (578h25m46.915585883s) [seg_pos={0 0}, upload=508543]: Begin upload segment... Feb 8 18:22:08.733181557 (578h25m46.915626537s) [seg_pos={0 0}, upload=508543]: Done begin upload segment. Feb 8 18:22:08.733209560 (578h25m46.91565454s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.734456150 (578h25m46.916901119s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735003412 (578h25m46.917448382s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.735547057 (578h25m46.917992027s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736016534 (578h25m46.918461504s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.736700663 (578h25m46.919145632s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.737249127 (578h25m46.919694097s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.738835811 (578h25m46.921280781s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.740483069 (578h25m46.922928039s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.742093397 (578h25m46.924538358s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.743414195 (578h25m46.925859165s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.744793662 (578h25m46.927238631s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.746142762 (578h25m46.928587733s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.747467858 (578h25m46.929912828s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.748772856 (578h25m46.931217826s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.750173122 (578h25m46.932618081s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Uploading piece... Feb 8 18:22:08.815099907 (578h25m46.997544877s) [upload=508543]: Next returned nil segment Feb 8 18:22:08.815114985 (578h25m46.997559955s) [upload=508543]: Waiting for error group managing segments... Feb 8 18:22:08.823973334 (578h25m47.006418305s) [noise=true , address=45.91.116.8:4080 , piece_id=X6TLA7EHHZQGT6SCQ4F7ZSIFZRW5SDQBCHYCCZJPLYRVKSEHRR7Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.824002930 (578h25m47.0064479s) [noise=true , address=79.127.219.46:4893 , piece_id=P46OGNKK54WVCD4FKWUDSLDMN6F46QVBOE5OFFS2K5BUSR6FFTKQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825289584 (578h25m47.007734554s) [noise=true , address=45.91.116.10:4084 , piece_id=Y4QUHP4E36IQCQWXKYPR3XRN2Y7AW3YH7LNPGMT7I3L422AZ52DA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825541855 (578h25m47.007986825s) [noise=true , address=79.127.219.38:4819 , piece_id=QQ3MGCUXIITV25A5XTIGD64TAMSZDSLWS6TSQCBVLWVXKMSJNUYA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.825663913 (578h25m47.008108872s) [noise=true , address=45.91.116.12:4228 , piece_id=UBR6VDMGYJMZIHU6DL7WLTF7YUVU4VB6NDPMXVQR725BV6LQC6OA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.827626879 (578h25m47.010071849s) [noise=true , address=79.127.219.41:4930 , piece_id=BLXQFDSFKZGLSCLMQE6QNGXSPI2MDSPZVIJFWLHFS5TDNKWJNQRQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.828093091 (578h25m47.010538061s) [noise=true , address=79.127.219.41:4928 , piece_id=WNKCF7E5B5WANQMPAL5XSB6XJSAGKHQOIPMJQ6FLOS2OCAS7FVBA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.841501550 (578h25m47.02394652s) [noise=true , address=79.127.219.46:4891 , piece_id=WVUHQNNL3JHVPEZH6VGNVISMRZX2QLVZSCNHWJXHAKMLYUQPGLSQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:08.921404384 (578h25m47.103849354s) [noise=true , address=79.127.219.45:4858 , piece_id=6ZZYBE4UMBCODMJ3TIJDJDTGV64RLH2JER2JTHVIETR5KGN5FFBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.355478860 (578h26m15.53792386s) [noise=true , address=77.90.31.13:4629 , piece_id=ZBFHKOPYSLIXFSYVYAJM4C327REGGTPNDV357MKN5FGYL4WPEN6Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:37.734320699 (578h26m15.916765698s) [noise=true , address=77.90.31.16:4736 , piece_id=7DNLJY5VM2EAXL4BZRSJSEPTGYOY2TW6QNDA566RIQQBRDIBGX3Q, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.048804032 (578h26m16.231249003s) [noise=true , address=77.90.31.10:4518 , piece_id=YBGNEXHHQAPIZSC7IWKDJ23BUVT445MXPUJU7WOQOSKLKTJKWJ3A, seg_pos={0 0}, upload=508543]: Done uploading piece. err: <nil> Feb 8 18:22:38.701872107 (578h26m16.884317076s) [noise=true , address=77.90.31.14:4661 , piece_id=OCIDEUBAR4PMI4CLLLI2UY2JET6HLUROUPZDWYQ3PZLJP5OFHIQA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702093940 (578h26m16.88453891s) [noise=true , address=77.90.31.11:4553 , piece_id=TITACQ37FLNIM4YDZN5BCU43Z4FC7EKIOWERUVUCMCCSL7SY3EBQ, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:58044->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702274628 (578h26m16.884719608s) [noise=true , address=77.90.31.17:4773 , piece_id=4G5QCXQTMHJGXONIVPLAEGSGYEO5Z5CCJ7PIOJD2DYF3QAXVFCLA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: read tcp 5.161.89.252:41666->77.90.31.17:4773: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Feb 8 18:22:38.702447421 (578h26m16.884892391s) [noise=true , address=77.90.31.17:4770 , piece_id=KDVMLGEGDC3Q7DIOCG6C4ROBDS7LZ56VF7ZCWBASN4RU5V5KVACA, seg_pos={0 0}, upload=508543]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:50134->77.90.31.17:4770: use of closed network connection; conn closed: op Feb 8 18:22:38.702472618 (578h26m16.884917588s) [seg_pos={0 0}, upload=508543]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Feb 8 18:22:38.730179540 (578h26m16.91262451s) [upload=508543]: Done uploading segments... Feb 8 18:22:38.730208785 (578h26m16.912653765s) [upload=508543]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 328.688173ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)