Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.............................X.....X...........X.X.XX.XXXXX.] duration: 22.295083535s FAIL (2025-08-26 05:16:16 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:15:11.856866547 (13h11m27.082049594s) [upload=11590]: Starting upload Aug 26 05:15:11.856947388 (13h11m27.082130445s) [upload=11590]: Done starting upload Aug 26 05:15:11.857007762 (13h11m27.082190808s) [upload=11590]: Uploading segments... Aug 26 05:15:11.857250268 (13h11m27.082433315s) [upload=11590]: Got next segment. Inline: false Aug 26 05:15:11.857286635 (13h11m27.082469692s) [upload=11590]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:15:11.857293298 (13h11m27.082476355s) [upload=11590]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:15:22.858314105 (13h11m38.083497183s) [seg_pos={0 0}, upload=11590]: Begin upload segment... Aug 26 05:15:22.858367536 (13h11m38.083550623s) [seg_pos={0 0}, upload=11590]: Done begin upload segment. Aug 26 05:15:22.858400479 (13h11m38.083583556s) [noise=true , address=45.91.116.14:4190 , piece_id=4R7WJUCTCWBXGOWT26AKRAJJJGVAWVOQWQDI5CUAXWAGKBO3LUSA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.858456674 (13h11m38.083639761s) [noise=true , address=77.90.31.15:4699 , piece_id=DK3JB4BSU5D4DGRDKIGEYH4J2XFGGHQGSYLDLG76OIILI5QQF44Q, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.858923801 (13h11m38.084106879s) [noise=true , address=45.91.116.5:4298 , piece_id=XABSOODUDZQAMVECL6IVUVWMQFZX3IAOZU7D4UHF3KIHP3B3CMPA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859079042 (13h11m38.08426213s) [noise=true , address=45.91.116.9:4261 , piece_id=PK26MWMBLSKNWB2PMTGGAF6NZ4EAY5BIUQP3CVGYOXRXI2NEWK4Q, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859314645 (13h11m38.084497722s) [noise=true , address=45.91.116.13:4155 , piece_id=2ZPF5WCMSCW3S2I2RC2EVJIDNFKH775TIE3J7Q3AJRHR5JJKBJ3A, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859514331 (13h11m38.084697417s) [noise=true , address=45.91.116.13:4154 , piece_id=UVDYGWF7HRYICI4YDHO2GEBAG5YJX24MFKIVGIVJMFAGKH6IQIOQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859739903 (13h11m38.084922981s) [noise=true , address=45.91.116.11:4118 , piece_id=XADWBNSBO7PIOWZROEFOVQVYZBY7DXARHZOYDIOLBA54ZPFU4NJQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860027705 (13h11m38.085210792s) [noise=true , address=104.36.82.7:4909 , piece_id=7YOQSRLJITDS42AURAM4BP24GSE42OPR4JCNLQ6GD6Y6M5ZZLMIQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860244512 (13h11m38.085427589s) [noise=true , address=77.90.31.13:4626 , piece_id=6OVHSO55T4UTUWVLDB4WT2QLK5U3JTIDI3ACVPYZB4LUHTBOZ3EQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860413158 (13h11m38.085596236s) [noise=true , address=45.91.116.9:4263 , piece_id=YPCUKYKUKEDWDZFRYMNCULCMUAOVLYA3DTZJU7KRFUGPQCE5Z6ZA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860662036 (13h11m38.085845133s) [noise=true , address=104.36.82.12:4150 , piece_id=Y4AJKPQYW5A2PSV5F6IZ27C6XE2BYU7F7OUWGNAQXDKCZPOTGDHA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860962371 (13h11m38.086145457s) [noise=true , address=77.90.31.21:4879 , piece_id=4I3TJN7NUHC2QOJ6JLLSXTM6SQ7UMEQCWYBIRNYNRAJYKENJBQQQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861160522 (13h11m38.0863436s) [noise=true , address=104.36.82.2:4729 , piece_id=YBU6CLRMUTMBQJMBTQ7J4YBXPDGDSAIKTY36HWSGBXOXTY6UPKGA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861340972 (13h11m38.086524019s) [noise=true , address=77.90.31.14:4663 , piece_id=RM3L6KVSXK7XPWIS2PB63PBSKC4XF22EKHY3PAJE264NNG65TRHQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861443835 (13h11m38.086626881s) [noise=true , address=45.91.116.4:4939 , piece_id=FKDXX5QA7VPHPQ5WW4W3OJCKBFISFTGZHQSOPCF2LKUZSFOJHLHA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861699475 (13h11m38.086882522s) [noise=true , address=77.90.31.11:4554 , piece_id=DXNENQZMA57JJ4QB46OSDDVN6RRTTT6DAWNJ3JTUWWHYL2XUA7GA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.990256495 (13h11m38.215439542s) [upload=11590]: Next returned nil segment Aug 26 05:15:22.990276573 (13h11m38.21545963s) [upload=11590]: Waiting for error group managing segments... Aug 26 05:15:22.998364304 (13h11m38.223547351s) [noise=true , address=45.91.116.9:4261 , piece_id=PK26MWMBLSKNWB2PMTGGAF6NZ4EAY5BIUQP3CVGYOXRXI2NEWK4Q, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.998564650 (13h11m38.223747697s) [noise=true , address=45.91.116.14:4190 , piece_id=4R7WJUCTCWBXGOWT26AKRAJJJGVAWVOQWQDI5CUAXWAGKBO3LUSA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.998988045 (13h11m38.224171092s) [noise=true , address=45.91.116.11:4118 , piece_id=XADWBNSBO7PIOWZROEFOVQVYZBY7DXARHZOYDIOLBA54ZPFU4NJQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.999143488 (13h11m38.224326535s) [noise=true , address=45.91.116.9:4263 , piece_id=YPCUKYKUKEDWDZFRYMNCULCMUAOVLYA3DTZJU7KRFUGPQCE5Z6ZA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.999571281 (13h11m38.224754328s) [noise=true , address=77.90.31.13:4626 , piece_id=6OVHSO55T4UTUWVLDB4WT2QLK5U3JTIDI3ACVPYZB4LUHTBOZ3EQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.999598783 (13h11m38.22478183s) [noise=true , address=45.91.116.13:4154 , piece_id=UVDYGWF7HRYICI4YDHO2GEBAG5YJX24MFKIVGIVJMFAGKH6IQIOQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000104903 (13h11m38.22528795s) [noise=true , address=45.91.116.5:4298 , piece_id=XABSOODUDZQAMVECL6IVUVWMQFZX3IAOZU7D4UHF3KIHP3B3CMPA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000133807 (13h11m38.225316864s) [noise=true , address=77.90.31.21:4879 , piece_id=4I3TJN7NUHC2QOJ6JLLSXTM6SQ7UMEQCWYBIRNYNRAJYKENJBQQQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000584494 (13h11m38.225767551s) [noise=true , address=45.91.116.13:4155 , piece_id=2ZPF5WCMSCW3S2I2RC2EVJIDNFKH775TIE3J7Q3AJRHR5JJKBJ3A, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000818082 (13h11m38.226001139s) [noise=true , address=45.91.116.4:4939 , piece_id=FKDXX5QA7VPHPQ5WW4W3OJCKBFISFTGZHQSOPCF2LKUZSFOJHLHA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001007669 (13h11m38.226190736s) [noise=true , address=104.36.82.7:4909 , piece_id=7YOQSRLJITDS42AURAM4BP24GSE42OPR4JCNLQ6GD6Y6M5ZZLMIQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001806589 (13h11m38.226989646s) [noise=true , address=77.90.31.14:4663 , piece_id=RM3L6KVSXK7XPWIS2PB63PBSKC4XF22EKHY3PAJE264NNG65TRHQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001962061 (13h11m38.227145108s) [noise=true , address=77.90.31.15:4699 , piece_id=DK3JB4BSU5D4DGRDKIGEYH4J2XFGGHQGSYLDLG76OIILI5QQF44Q, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001984242 (13h11m38.22716732s) [seg_pos={0 0}, upload=11590]: Segment reached optimal threshold of 13 pieces. Aug 26 05:15:23.002128384 (13h11m38.227311431s) [noise=true , address=104.36.82.12:4150 , piece_id=Y4AJKPQYW5A2PSV5F6IZ27C6XE2BYU7F7OUWGNAQXDKCZPOTGDHA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Aug 26 05:15:23.002141278 (13h11m38.227324335s) [noise=true , address=77.90.31.11:4554 , piece_id=DXNENQZMA57JJ4QB46OSDDVN6RRTTT6DAWNJ3JTUWWHYL2XUA7GA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 26 05:15:23.002152920 (13h11m38.227335967s) [noise=true , address=104.36.82.2:4729 , piece_id=YBU6CLRMUTMBQJMBTQ7J4YBXPDGDSAIKTY36HWSGBXOXTY6UPKGA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Aug 26 05:15:23.002175222 (13h11m38.227358269s) [seg_pos={0 0}, upload=11590]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:15:23.002254370 (13h11m38.227437437s) [upload=11590]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:15:23.002260071 (13h11m38.227443118s) [upload=11590]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:15:31.060086659 (13h11m46.285269727s) [upload=11590]: Done uploading segments... Aug 26 05:15:31.060104984 (13h11m46.285288061s) [upload=11590]: Upload finished. err: <nil> Duration: 22.295109003s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................................X........XX.X.X.X.] duration: 44.392562624s FAIL (2025-08-26 05:16:13 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 0704832 (13h11m2.905887889s) [upload=11582]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:14:47.902776638 (13h11m3.127959686s) [seg_pos={0 0}, upload=11582]: Begin upload segment... Aug 26 05:14:47.902820661 (13h11m3.128003709s) [seg_pos={0 0}, upload=11582]: Done begin upload segment. Aug 26 05:14:47.902849054 (13h11m3.128032111s) [noise=true , address=45.91.116.11:4146 , piece_id=HIWPDZR3CTF22YZCYH73QMJFSGET7QTBHRMNOLVMTSU6RJXSWTFA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.902873490 (13h11m3.128056537s) [noise=true , address=77.90.31.14:4694 , piece_id=O6K4VOHGODJGHAKXAGEKEAXU7ZXHIEVTE2S6KBKWSUJVZKACUWNQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903189514 (13h11m3.128372571s) [noise=true , address=104.36.82.13:4214 , piece_id=MBC7UIGS4Y7LUYQKQTG33CMKX7O25ANAV73ONFHCWKX7MLDIXOVA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903342101 (13h11m3.128525148s) [noise=true , address=45.91.116.14:4203 , piece_id=C4YPVPOP56QR7P6RLTPZYXDBOJGNVSI7QS52DQRKESU6MBUEXN2A, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903413926 (13h11m3.128596973s) [noise=true , address=104.36.82.50:4030 , piece_id=FIQJULHAAWQQXVV3BTH5XIIYNUIVLOPXVEEOJWETTY6KBWKCX2WQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903641763 (13h11m3.12882482s) [noise=true , address=79.127.205.246:4268 , piece_id=WMRJ2R3PWW4PEJKMI4WO6RDGV24VBMFDJMNA5ST4XXWNG3EZGYYA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903697358 (13h11m3.128880406s) [noise=true , address=104.36.82.44:4873 , piece_id=CVFBS5TPDC7DSBY2MGZH4EL5DO3PLJDE2GJCZ2TL6JV7ETU2E34A, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.904382555 (13h11m3.129565612s) [noise=true , address=77.90.31.19:4855 , piece_id=V36EQSYCAE3O2GFK4KNVLKSYOJADKRIOMRBVSOHRI3RNL3AFEYNA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.904391010 (13h11m3.129574068s) [noise=true , address=104.36.82.6:4891 , piece_id=HBARHJPGMLXHPEEA27KEMNMJVRFWTWR74YQL5XTO7KYRZ266VJXA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.904787035 (13h11m3.129970082s) [noise=true , address=104.36.82.60:4354 , piece_id=HFCYUWFHQ6AJA7REULM6KRFPMDFBBKRRJ22VQXSVTM5S4X3BX3PA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.905089312 (13h11m3.130272359s) [noise=true , address=104.36.82.49:4987 , piece_id=ECBZZ23GJA5TWTPB2X7GZBFXIY5VL5FJR77KBF64QCO5BEVUALCQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.905389606 (13h11m3.130572683s) [noise=true , address=104.36.82.10:4085 , piece_id=AZBDQYQUON4UVMJ3F4ULQXTNBB2OSWFFC72KU3MA2J7RGBJZELWA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.905880078 (13h11m3.131063125s) [noise=true , address=79.127.201.215:4349 , piece_id=BU6KUUSE77HTCME4DFHPPMADIFAFHI43PL2JPDEHZHZEYJMRXR4Q, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.906031132 (13h11m3.131214179s) [noise=true , address=45.91.116.7:4030 , piece_id=JX6NWRZODJ3PEDOU43U5KQXHYBFYBPYJNDISY2T3DLK5NYZ5I7GA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.906575734 (13h11m3.131758791s) [noise=true , address=79.127.205.245:4244 , piece_id=G25ULKCGLHJCT7QCOP6ZSPXA7CHOS3KNIRZMRQT6NWRZPVRTCZCQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.906838247 (13h11m3.132021314s) [noise=true , address=45.91.116.6:4992 , piece_id=6G7PY5YY32FTZBPRVLKSHSUH7KNYRIGHAFOPT7AVBCBM7HYH3MVQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.907199135 (13h11m3.132382192s) [noise=true , address=45.91.116.13:4169 , piece_id=7ND7C76WDTB77MRO3RPSQDGQFOK7H5WOBSD7RCVI7OUA5H4HSSXA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.910863127 (13h11m3.136046194s) [noise=true , address=104.36.82.23:4456 , piece_id=NOIRLI7AVEW4P3BRPJI27WOSFOO2WP2E6RDPFWWGHEWUKZ7HTF7Q, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.996930147 (13h11m3.222113204s) [upload=11582]: Next returned nil segment Aug 26 05:14:47.996953811 (13h11m3.222136868s) [upload=11582]: Waiting for error group managing segments... Aug 26 05:14:48.004359973 (13h11m3.229543019s) [noise=true , address=45.91.116.7:4030 , piece_id=JX6NWRZODJ3PEDOU43U5KQXHYBFYBPYJNDISY2T3DLK5NYZ5I7GA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.005508650 (13h11m3.230691697s) [noise=true , address=45.91.116.11:4146 , piece_id=HIWPDZR3CTF22YZCYH73QMJFSGET7QTBHRMNOLVMTSU6RJXSWTFA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.005522095 (13h11m3.230705142s) [noise=true , address=45.91.116.14:4203 , piece_id=C4YPVPOP56QR7P6RLTPZYXDBOJGNVSI7QS52DQRKESU6MBUEXN2A, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.006008057 (13h11m3.231191105s) [noise=true , address=45.91.116.13:4169 , piece_id=7ND7C76WDTB77MRO3RPSQDGQFOK7H5WOBSD7RCVI7OUA5H4HSSXA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.008257333 (13h11m3.23344037s) [noise=true , address=104.36.82.49:4987 , piece_id=ECBZZ23GJA5TWTPB2X7GZBFXIY5VL5FJR77KBF64QCO5BEVUALCQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.008662254 (13h11m3.233845311s) [noise=true , address=104.36.82.44:4873 , piece_id=CVFBS5TPDC7DSBY2MGZH4EL5DO3PLJDE2GJCZ2TL6JV7ETU2E34A, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.009403165 (13h11m3.234586212s) [noise=true , address=77.90.31.14:4694 , piece_id=O6K4VOHGODJGHAKXAGEKEAXU7ZXHIEVTE2S6KBKWSUJVZKACUWNQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.009804760 (13h11m3.234987806s) [noise=true , address=77.90.31.19:4855 , piece_id=V36EQSYCAE3O2GFK4KNVLKSYOJADKRIOMRBVSOHRI3RNL3AFEYNA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.011645477 (13h11m3.236828524s) [noise=true , address=104.36.82.13:4214 , piece_id=MBC7UIGS4Y7LUYQKQTG33CMKX7O25ANAV73ONFHCWKX7MLDIXOVA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.016428189 (13h11m3.241611427s) [noise=true , address=104.36.82.50:4030 , piece_id=FIQJULHAAWQQXVV3BTH5XIIYNUIVLOPXVEEOJWETTY6KBWKCX2WQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.016598520 (13h11m3.241781567s) [noise=true , address=104.36.82.10:4085 , piece_id=AZBDQYQUON4UVMJ3F4ULQXTNBB2OSWFFC72KU3MA2J7RGBJZELWA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.017570516 (13h11m3.242753553s) [noise=true , address=104.36.82.6:4891 , piece_id=HBARHJPGMLXHPEEA27KEMNMJVRFWTWR74YQL5XTO7KYRZ266VJXA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.030796789 (13h11m3.255979836s) [noise=true , address=104.36.82.60:4354 , piece_id=HFCYUWFHQ6AJA7REULM6KRFPMDFBBKRRJ22VQXSVTM5S4X3BX3PA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.089222399 (13h11m3.314405456s) [noise=true , address=104.36.82.23:4456 , piece_id=NOIRLI7AVEW4P3BRPJI27WOSFOO2WP2E6RDPFWWGHEWUKZ7HTF7Q, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.089247306 (13h11m3.314430354s) [seg_pos={0 0}, upload=11582]: Segment reached optimal threshold of 14 pieces. Aug 26 05:14:48.089472019 (13h11m3.314655076s) [noise=true , address=45.91.116.6:4992 , piece_id=6G7PY5YY32FTZBPRVLKSHSUH7KNYRIGHAFOPT7AVBCBM7HYH3MVQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ai9G3ULmwAy4zkKPUonML5fgxZGvJJgmqQeGfYHWqUKpehgtx): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:48.089527372 (13h11m3.314710429s) [noise=true , address=79.127.201.215:4349 , piece_id=BU6KUUSE77HTCME4DFHPPMADIFAFHI43PL2JPDEHZHZEYJMRXR4Q, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: write tcp 5.161.89.252:45582->79.127.201.215:4349: use of closed network connection; write tcp 5.161.89.252:45582->79.127.201.215:4349: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45582->79.127.201.215:4349: use of closed network connection Aug 26 05:14:48.089730384 (13h11m3.314913441s) [noise=true , address=79.127.205.246:4268 , piece_id=WMRJ2R3PWW4PEJKMI4WO6RDGV24VBMFDJMNA5ST4XXWNG3EZGYYA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k72iHWdcnM2JKs7VPTQFQwmpQ835HmAP8KpXxjMbEy5QxfzG): protocol: read tcp 5.161.89.252:10486->79.127.205.246:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:14:48.089765280 (13h11m3.314948327s) [noise=true , address=79.127.205.245:4244 , piece_id=G25ULKCGLHJCT7QCOP6ZSPXA7CHOS3KNIRZMRQT6NWRZPVRTCZCQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JFy3MxMFZYnjxJmujGpmcTdpVz6qLUMKf1aB3eYW236kCYgng): protocol: read tcp 5.161.89.252:25038->79.127.205.245:4244: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:14:48.089773956 (13h11m3.314957003s) [seg_pos={0 0}, upload=11582]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:14:48.089836283 (13h11m3.31501934s) [upload=11582]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:14:48.089842094 (13h11m3.315025141s) [upload=11582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:48.256048660 (13h11m3.481231707s) [upload=11582]: Done uploading segments... Aug 26 05:14:48.256071674 (13h11m3.481254731s) [upload=11582]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 44.392600746s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.................................................X.....XX.XX] duration: 329.452181ms remote: "tcp:45.38.24.2:443" response headers: Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=185F394AB595C12F X-Request-Id=N1znMTMT8Va X-Xss-Protection=1; mode=block Content-Length=433 Content-Security-Policy=block-all-mixed-content Date=Tue, 26 Aug 2025 05:16:13 GMT target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:16:13 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F394AB595C12F</RequestId><HostId></HostId></Error>" Duration: 329.483139ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [..............................XX.......X..........XX.X.X.XX.] duration: 18.958280248s FAIL (2025-08-26 05:16:01 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:14:29.029036501 (13h10m44.254219628s) [upload=11579]: Starting upload Aug 26 05:14:29.029135167 (13h10m44.254318244s) [upload=11579]: Done starting upload Aug 26 05:14:29.029207823 (13h10m44.25439091s) [upload=11579]: Uploading segments... Aug 26 05:14:29.029253299 (13h10m44.254436376s) [upload=11579]: Got next segment. Inline: true Aug 26 05:14:29.029259290 (13h10m44.254442367s) [upload=11579]: Waiting for error group managing segments... Aug 26 05:14:29.029307520 (13h10m44.254490598s) [upload=11579]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:14:29.029313171 (13h10m44.254496248s) [upload=11579]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:14:29.029317299 (13h10m44.254500376s) [upload=11579]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:41.554926683 (13h10m56.78010974s) [upload=11579]: Done uploading segments... Aug 26 05:14:41.554940789 (13h10m56.780123846s) [upload=11579]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 18.95832372s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [...................................................X.X..XX..] duration: 16.663113769s FAIL (2025-08-26 05:16:01 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:13:43.987058282 (13h9m59.212241339s) [upload=11567]: Starting upload Aug 26 05:13:43.987144814 (13h9m59.212327871s) [upload=11567]: Done starting upload Aug 26 05:13:43.987212292 (13h9m59.212395349s) [upload=11567]: Uploading segments... Aug 26 05:13:43.987257306 (13h9m59.212440363s) [upload=11567]: Got next segment. Inline: true Aug 26 05:13:43.987263478 (13h9m59.212446525s) [upload=11567]: Waiting for error group managing segments... Aug 26 05:13:43.987304054 (13h9m59.212487152s) [upload=11567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:43.987309574 (13h9m59.212492631s) [upload=11567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:13:43.987313713 (13h9m59.21249677s) [upload=11567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:09.912697999 (13h10m25.137881087s) [upload=11567]: Done uploading segments... Aug 26 05:14:09.912726052 (13h10m25.137909209s) [upload=11567]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 16.663140237s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X.X..X...] duration: 17.106425989s FAIL (2025-08-26 05:16:01 UTC) operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3927EAB52717, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 17.106459172s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 4 out of 60 is failed (threshold=6) [...................................X...............X..X....X] duration: 364.20219ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:15:54 GMT Accept-Ranges=bytes X-Request-Id=PzQG5qDgV69 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:15:54 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 364.229562ms edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............................X..................XXX....XXX.] duration: 17.613267477s FAIL (2025-08-26 05:15:53 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F393D8A8A6220, HostID: , api error SlowDown: Please reduce your request rate. Duration: 17.613324594s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..................................X....XX.........XX.XX..XXX] duration: 5.696295278s FAIL (2025-08-26 05:15:48 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3945025C03C0, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 5.696314324s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X..........X....X..] duration: 5.781827596s remote: "tcp:136.0.77.2:443" response headers: X-Request-Id=VcCcTMkH9av Content-Length=423 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=185F392FFCED1105 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:14:18 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:15:35 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F392FFCED1105</RequestId><HostId></HostId></Error>" Duration: 5.78184589s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..............................X.....X...........X.X.XX.XXXXX] duration: 23.118463895s FAIL (2025-08-26 05:15:34 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:15:11.856866547 (13h11m27.082049594s) [upload=11590]: Starting upload Aug 26 05:15:11.856947388 (13h11m27.082130445s) [upload=11590]: Done starting upload Aug 26 05:15:11.857007762 (13h11m27.082190808s) [upload=11590]: Uploading segments... Aug 26 05:15:11.857250268 (13h11m27.082433315s) [upload=11590]: Got next segment. Inline: false Aug 26 05:15:11.857286635 (13h11m27.082469692s) [upload=11590]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:15:11.857293298 (13h11m27.082476355s) [upload=11590]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:15:22.858314105 (13h11m38.083497183s) [seg_pos={0 0}, upload=11590]: Begin upload segment... Aug 26 05:15:22.858367536 (13h11m38.083550623s) [seg_pos={0 0}, upload=11590]: Done begin upload segment. Aug 26 05:15:22.858400479 (13h11m38.083583556s) [noise=true , address=45.91.116.14:4190 , piece_id=4R7WJUCTCWBXGOWT26AKRAJJJGVAWVOQWQDI5CUAXWAGKBO3LUSA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.858456674 (13h11m38.083639761s) [noise=true , address=77.90.31.15:4699 , piece_id=DK3JB4BSU5D4DGRDKIGEYH4J2XFGGHQGSYLDLG76OIILI5QQF44Q, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.858923801 (13h11m38.084106879s) [noise=true , address=45.91.116.5:4298 , piece_id=XABSOODUDZQAMVECL6IVUVWMQFZX3IAOZU7D4UHF3KIHP3B3CMPA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859079042 (13h11m38.08426213s) [noise=true , address=45.91.116.9:4261 , piece_id=PK26MWMBLSKNWB2PMTGGAF6NZ4EAY5BIUQP3CVGYOXRXI2NEWK4Q, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859314645 (13h11m38.084497722s) [noise=true , address=45.91.116.13:4155 , piece_id=2ZPF5WCMSCW3S2I2RC2EVJIDNFKH775TIE3J7Q3AJRHR5JJKBJ3A, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859514331 (13h11m38.084697417s) [noise=true , address=45.91.116.13:4154 , piece_id=UVDYGWF7HRYICI4YDHO2GEBAG5YJX24MFKIVGIVJMFAGKH6IQIOQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.859739903 (13h11m38.084922981s) [noise=true , address=45.91.116.11:4118 , piece_id=XADWBNSBO7PIOWZROEFOVQVYZBY7DXARHZOYDIOLBA54ZPFU4NJQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860027705 (13h11m38.085210792s) [noise=true , address=104.36.82.7:4909 , piece_id=7YOQSRLJITDS42AURAM4BP24GSE42OPR4JCNLQ6GD6Y6M5ZZLMIQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860244512 (13h11m38.085427589s) [noise=true , address=77.90.31.13:4626 , piece_id=6OVHSO55T4UTUWVLDB4WT2QLK5U3JTIDI3ACVPYZB4LUHTBOZ3EQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860413158 (13h11m38.085596236s) [noise=true , address=45.91.116.9:4263 , piece_id=YPCUKYKUKEDWDZFRYMNCULCMUAOVLYA3DTZJU7KRFUGPQCE5Z6ZA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860662036 (13h11m38.085845133s) [noise=true , address=104.36.82.12:4150 , piece_id=Y4AJKPQYW5A2PSV5F6IZ27C6XE2BYU7F7OUWGNAQXDKCZPOTGDHA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.860962371 (13h11m38.086145457s) [noise=true , address=77.90.31.21:4879 , piece_id=4I3TJN7NUHC2QOJ6JLLSXTM6SQ7UMEQCWYBIRNYNRAJYKENJBQQQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861160522 (13h11m38.0863436s) [noise=true , address=104.36.82.2:4729 , piece_id=YBU6CLRMUTMBQJMBTQ7J4YBXPDGDSAIKTY36HWSGBXOXTY6UPKGA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861340972 (13h11m38.086524019s) [noise=true , address=77.90.31.14:4663 , piece_id=RM3L6KVSXK7XPWIS2PB63PBSKC4XF22EKHY3PAJE264NNG65TRHQ, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861443835 (13h11m38.086626881s) [noise=true , address=45.91.116.4:4939 , piece_id=FKDXX5QA7VPHPQ5WW4W3OJCKBFISFTGZHQSOPCF2LKUZSFOJHLHA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.861699475 (13h11m38.086882522s) [noise=true , address=77.90.31.11:4554 , piece_id=DXNENQZMA57JJ4QB46OSDDVN6RRTTT6DAWNJ3JTUWWHYL2XUA7GA, seg_pos={0 0}, upload=11590]: Uploading piece... Aug 26 05:15:22.990256495 (13h11m38.215439542s) [upload=11590]: Next returned nil segment Aug 26 05:15:22.990276573 (13h11m38.21545963s) [upload=11590]: Waiting for error group managing segments... Aug 26 05:15:22.998364304 (13h11m38.223547351s) [noise=true , address=45.91.116.9:4261 , piece_id=PK26MWMBLSKNWB2PMTGGAF6NZ4EAY5BIUQP3CVGYOXRXI2NEWK4Q, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.998564650 (13h11m38.223747697s) [noise=true , address=45.91.116.14:4190 , piece_id=4R7WJUCTCWBXGOWT26AKRAJJJGVAWVOQWQDI5CUAXWAGKBO3LUSA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.998988045 (13h11m38.224171092s) [noise=true , address=45.91.116.11:4118 , piece_id=XADWBNSBO7PIOWZROEFOVQVYZBY7DXARHZOYDIOLBA54ZPFU4NJQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.999143488 (13h11m38.224326535s) [noise=true , address=45.91.116.9:4263 , piece_id=YPCUKYKUKEDWDZFRYMNCULCMUAOVLYA3DTZJU7KRFUGPQCE5Z6ZA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.999571281 (13h11m38.224754328s) [noise=true , address=77.90.31.13:4626 , piece_id=6OVHSO55T4UTUWVLDB4WT2QLK5U3JTIDI3ACVPYZB4LUHTBOZ3EQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:22.999598783 (13h11m38.22478183s) [noise=true , address=45.91.116.13:4154 , piece_id=UVDYGWF7HRYICI4YDHO2GEBAG5YJX24MFKIVGIVJMFAGKH6IQIOQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000104903 (13h11m38.22528795s) [noise=true , address=45.91.116.5:4298 , piece_id=XABSOODUDZQAMVECL6IVUVWMQFZX3IAOZU7D4UHF3KIHP3B3CMPA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000133807 (13h11m38.225316864s) [noise=true , address=77.90.31.21:4879 , piece_id=4I3TJN7NUHC2QOJ6JLLSXTM6SQ7UMEQCWYBIRNYNRAJYKENJBQQQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000584494 (13h11m38.225767551s) [noise=true , address=45.91.116.13:4155 , piece_id=2ZPF5WCMSCW3S2I2RC2EVJIDNFKH775TIE3J7Q3AJRHR5JJKBJ3A, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.000818082 (13h11m38.226001139s) [noise=true , address=45.91.116.4:4939 , piece_id=FKDXX5QA7VPHPQ5WW4W3OJCKBFISFTGZHQSOPCF2LKUZSFOJHLHA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001007669 (13h11m38.226190736s) [noise=true , address=104.36.82.7:4909 , piece_id=7YOQSRLJITDS42AURAM4BP24GSE42OPR4JCNLQ6GD6Y6M5ZZLMIQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001806589 (13h11m38.226989646s) [noise=true , address=77.90.31.14:4663 , piece_id=RM3L6KVSXK7XPWIS2PB63PBSKC4XF22EKHY3PAJE264NNG65TRHQ, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001962061 (13h11m38.227145108s) [noise=true , address=77.90.31.15:4699 , piece_id=DK3JB4BSU5D4DGRDKIGEYH4J2XFGGHQGSYLDLG76OIILI5QQF44Q, seg_pos={0 0}, upload=11590]: Done uploading piece. err: <nil> Aug 26 05:15:23.001984242 (13h11m38.22716732s) [seg_pos={0 0}, upload=11590]: Segment reached optimal threshold of 13 pieces. Aug 26 05:15:23.002128384 (13h11m38.227311431s) [noise=true , address=104.36.82.12:4150 , piece_id=Y4AJKPQYW5A2PSV5F6IZ27C6XE2BYU7F7OUWGNAQXDKCZPOTGDHA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Aug 26 05:15:23.002141278 (13h11m38.227324335s) [noise=true , address=77.90.31.11:4554 , piece_id=DXNENQZMA57JJ4QB46OSDDVN6RRTTT6DAWNJ3JTUWWHYL2XUA7GA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 26 05:15:23.002152920 (13h11m38.227335967s) [noise=true , address=104.36.82.2:4729 , piece_id=YBU6CLRMUTMBQJMBTQ7J4YBXPDGDSAIKTY36HWSGBXOXTY6UPKGA, seg_pos={0 0}, upload=11590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR): protocol: expected piece hash; context canceled; EOF Aug 26 05:15:23.002175222 (13h11m38.227358269s) [seg_pos={0 0}, upload=11590]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:15:23.002254370 (13h11m38.227437437s) [upload=11590]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:15:23.002260071 (13h11m38.227443118s) [upload=11590]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:15:31.060086659 (13h11m46.285269727s) [upload=11590]: Done uploading segments... Aug 26 05:15:31.060104984 (13h11m46.285288061s) [upload=11590]: Upload finished. err: <nil> Duration: 23.118509811s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.............................XX.......X..X.......XXXXXXXX.XX] duration: 1.268294849s FAIL (2025-08-26 05:15:30 UTC) operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 1.268318113s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [..................................................X.....XX.X] duration: 225.69242ms remote: "tcp:45.38.24.2:443" response headers: Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F39402FDCB14C X-Request-Id=JXEuYW6gE9 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:15:27 GMT target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:15:27 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F39402FDCB14C</RequestId><HostId></HostId></Error>" Duration: 225.725552ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 3 out of 60 is failed (threshold=6) [....................................X...............X..X....] duration: 29.717140426s remote: "tcp:136.0.77.2:443" response headers: Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:12:16 GMT X-Request-Id=cG2LXHRGapy Content-Type=text/html; charset=utf-8 target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:15:22 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 29.717159813s edge linksharing bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X.X..X..] duration: 49.978909964s FAIL (2025-08-26 05:15:19 UTC) operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3927EAB52717, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 49.978927659s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............................X..................XXX....XXX] duration: 3.940842966s FAIL (2025-08-26 05:15:16 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F393D8A8A6220, HostID: , api error SlowDown: Please reduce your request rate. Duration: 3.940865038s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [..............................X.....XX..........XX....X..X.X] duration: 14.481742591s FAIL (2025-08-26 05:15:15 UTC) operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 14.481766136s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [....................................................X.X..XX.] duration: 19.754500912s FAIL (2025-08-26 05:15:14 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:13:43.987058282 (13h9m59.212241339s) [upload=11567]: Starting upload Aug 26 05:13:43.987144814 (13h9m59.212327871s) [upload=11567]: Done starting upload Aug 26 05:13:43.987212292 (13h9m59.212395349s) [upload=11567]: Uploading segments... Aug 26 05:13:43.987257306 (13h9m59.212440363s) [upload=11567]: Got next segment. Inline: true Aug 26 05:13:43.987263478 (13h9m59.212446525s) [upload=11567]: Waiting for error group managing segments... Aug 26 05:13:43.987304054 (13h9m59.212487152s) [upload=11567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:43.987309574 (13h9m59.212492631s) [upload=11567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:13:43.987313713 (13h9m59.21249677s) [upload=11567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:09.912697999 (13h10m25.137881087s) [upload=11567]: Done uploading segments... Aug 26 05:14:09.912726052 (13h10m25.137909209s) [upload=11567]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 19.754525808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..............................XX.......X..X.......XXXXXXXX.X] duration: 2.723879081s FAIL (2025-08-26 05:15:13 UTC) operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 2.723905331s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...............................X.....X...........X.X.XX.XXXX] duration: 30.001999367s FAIL (2025-08-26 05:15:11 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:14:41.805232867 (13h10m57.030415945s) [upload=11580]: Starting upload Aug 26 05:14:41.805355138 (13h10m57.030538215s) [upload=11580]: Done starting upload Aug 26 05:14:41.805530647 (13h10m57.030713724s) [upload=11580]: Uploading segments... Aug 26 05:14:41.805924637 (13h10m57.031107724s) [upload=11580]: Got next segment. Inline: false Aug 26 05:14:41.805985521 (13h10m57.031168598s) [upload=11580]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:14:41.805994588 (13h10m57.031177665s) [upload=11580]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:14:45.834560941 (13h11m1.059744048s) [seg_pos={0 0}, upload=11580]: Begin upload segment... Aug 26 05:14:45.834622616 (13h11m1.059805694s) [seg_pos={0 0}, upload=11580]: Done begin upload segment. Aug 26 05:14:45.834661660 (13h11m1.059844707s) [noise=true , address=45.91.116.12:4225 , piece_id=ZB5RTVXB6D62QIWTGKOXZEHX7E56HN7EUGHVYXUW2STTIIUNC3TQ, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.834677409 (13h11m1.059860487s) [noise=true , address=45.91.116.14:4189 , piece_id=TNPIJR536M4P35UOR564GINEHWHOFLWNWAJYL6GEBGVFUR7TBK4Q, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.835097489 (13h11m1.060280535s) [noise=true , address=104.36.82.7:4910 , piece_id=4A3M24FEVBPGGWIRPDYBWI5RDLCZ67MEZDRFHCTEIRWGFJFYIV3Q, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.835245647 (13h11m1.060428724s) [noise=true , address=77.90.31.19:4841 , piece_id=KWN3RSDH4ZQJS6ZP6UD2W2Y5WO6DHJIKINXB4ASM6XFSTHFUYVRQ, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.835331067 (13h11m1.060514124s) [noise=true , address=45.91.116.13:4155 , piece_id=3H5ULSQON2VH75EAZ2CTSVGGFKOXIJN3YH3DFYPD36TB5VQ5VU3A, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.835580797 (13h11m1.060763843s) [noise=true , address=104.36.82.4:4803 , piece_id=COG5ODRN7S3EYPOP6STDOOGGZZ5XIUCTN3XTJZYECELKMEZAPMDA, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.835683249 (13h11m1.060866326s) [noise=true , address=77.90.31.12:4590 , piece_id=TEGJLILYA3T3EVRGVXGGDUNZ737LZUMFKLU6TLD5FIYNNG6LH7VA, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.836105472 (13h11m1.061288548s) [noise=true , address=77.90.31.13:4628 , piece_id=32DIVWYH5UXVGJ5AFXGY5SBIHRRKKBXBSGJNABL2F76WFCU3MJKA, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.836680352 (13h11m1.061863429s) [noise=true , address=45.91.116.8:4048 , piece_id=3E7RZCG3F7BYXB2HSHBCO2GNH2DN2HIUEPOW7A3VIHF2WB3KFKMQ, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.837077668 (13h11m1.062260745s) [noise=true , address=45.91.116.7:4010 , piece_id=A7N6SORMGW27NZVXUQFCBY2LMRCNBJSXLFFWFZAVMRRI36BBGLPQ, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.837606390 (13h11m1.062789468s) [noise=true , address=45.91.116.4:4939 , piece_id=PM4NCKV4X7QSCALY4LRIBCYTLABM775VVEVZUMLSFUD3NH655HRA, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.838047258 (13h11m1.063230336s) [noise=true , address=45.91.116.6:4974 , piece_id=KL4FYCQPOQZHH2IPAG4WHXP2AWNAJJIMXO5NZBJAYVLTSGZ6TB6A, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.838474421 (13h11m1.063657499s) [noise=true , address=77.90.31.10:4520 , piece_id=HC6MLZGNL63RSWDXO6IF6N3DIU36DQQRAFFWWZEOMEA2EIB3JW6A, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.838866668 (13h11m1.064049746s) [noise=true , address=77.90.31.19:4842 , piece_id=7PDG754NNFDPX7HX5GZGAL76XKS3PASWSUTDY6ZQBRLXOSN4W7AQ, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.839285464 (13h11m1.064468542s) [noise=true , address=45.91.116.8:4045 , piece_id=J3S6NZBJJ77LQRZGA2DYCTBZDTXLTWUGP7AF5L5FD3VWJO56CKNQ, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.839680818 (13h11m1.064863925s) [noise=true , address=45.91.116.6:4973 , piece_id=N47XIYSWK25DIZABUVWQQY2IOKUIW5GFMVWHBIWQ6BVHMFQBFY5A, seg_pos={0 0}, upload=11580]: Uploading piece... Aug 26 05:14:45.934097160 (13h11m1.159280227s) [upload=11580]: Next returned nil segment Aug 26 05:14:45.934117618 (13h11m1.159300715s) [upload=11580]: Waiting for error group managing segments... Aug 26 05:14:45.943404662 (13h11m1.168587729s) [noise=true , address=45.91.116.7:4010 , piece_id=A7N6SORMGW27NZVXUQFCBY2LMRCNBJSXLFFWFZAVMRRI36BBGLPQ, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.943557759 (13h11m1.168740836s) [noise=true , address=45.91.116.8:4045 , piece_id=J3S6NZBJJ77LQRZGA2DYCTBZDTXLTWUGP7AF5L5FD3VWJO56CKNQ, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.943877960 (13h11m1.169061017s) [noise=true , address=77.90.31.12:4590 , piece_id=TEGJLILYA3T3EVRGVXGGDUNZ737LZUMFKLU6TLD5FIYNNG6LH7VA, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.944347183 (13h11m1.16953024s) [noise=true , address=45.91.116.6:4973 , piece_id=N47XIYSWK25DIZABUVWQQY2IOKUIW5GFMVWHBIWQ6BVHMFQBFY5A, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.944701278 (13h11m1.169884355s) [noise=true , address=77.90.31.10:4520 , piece_id=HC6MLZGNL63RSWDXO6IF6N3DIU36DQQRAFFWWZEOMEA2EIB3JW6A, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.944826553 (13h11m1.17000962s) [noise=true , address=45.91.116.12:4225 , piece_id=ZB5RTVXB6D62QIWTGKOXZEHX7E56HN7EUGHVYXUW2STTIIUNC3TQ, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.945357529 (13h11m1.170540596s) [noise=true , address=45.91.116.8:4048 , piece_id=3E7RZCG3F7BYXB2HSHBCO2GNH2DN2HIUEPOW7A3VIHF2WB3KFKMQ, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.945643346 (13h11m1.170826434s) [noise=true , address=77.90.31.19:4841 , piece_id=KWN3RSDH4ZQJS6ZP6UD2W2Y5WO6DHJIKINXB4ASM6XFSTHFUYVRQ, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.945834265 (13h11m1.171017332s) [noise=true , address=77.90.31.13:4628 , piece_id=32DIVWYH5UXVGJ5AFXGY5SBIHRRKKBXBSGJNABL2F76WFCU3MJKA, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.946317463 (13h11m1.17150053s) [noise=true , address=45.91.116.4:4939 , piece_id=PM4NCKV4X7QSCALY4LRIBCYTLABM775VVEVZUMLSFUD3NH655HRA, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.946340727 (13h11m1.171523803s) [noise=true , address=45.91.116.6:4974 , piece_id=KL4FYCQPOQZHH2IPAG4WHXP2AWNAJJIMXO5NZBJAYVLTSGZ6TB6A, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.946776415 (13h11m1.171959472s) [noise=true , address=104.36.82.7:4910 , piece_id=4A3M24FEVBPGGWIRPDYBWI5RDLCZ67MEZDRFHCTEIRWGFJFYIV3Q, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.946987752 (13h11m1.172170829s) [noise=true , address=77.90.31.19:4842 , piece_id=7PDG754NNFDPX7HX5GZGAL76XKS3PASWSUTDY6ZQBRLXOSN4W7AQ, seg_pos={0 0}, upload=11580]: Done uploading piece. err: <nil> Aug 26 05:14:45.947014803 (13h11m1.17219788s) [seg_pos={0 0}, upload=11580]: Segment reached optimal threshold of 13 pieces. Aug 26 05:14:45.947160857 (13h11m1.172343914s) [noise=true , address=45.91.116.13:4155 , piece_id=3H5ULSQON2VH75EAZ2CTSVGGFKOXIJN3YH3DFYPD36TB5VQ5VU3A, seg_pos={0 0}, upload=11580]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:45.947171647 (13h11m1.172354724s) [noise=true , address=45.91.116.14:4189 , piece_id=TNPIJR536M4P35UOR564GINEHWHOFLWNWAJYL6GEBGVFUR7TBK4Q, seg_pos={0 0}, upload=11580]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:45.947239265 (13h11m1.172422342s) [noise=true , address=104.36.82.4:4803 , piece_id=COG5ODRN7S3EYPOP6STDOOGGZZ5XIUCTN3XTJZYECELKMEZAPMDA, seg_pos={0 0}, upload=11580]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:45.947310098 (13h11m1.172493175s) [seg_pos={0 0}, upload=11580]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:14:45.947439631 (13h11m1.172622707s) [upload=11580]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:14:45.947445702 (13h11m1.172628779s) [upload=11580]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:53.995268685 (13h11m9.220451722s) [upload=11580]: Done uploading segments... Aug 26 05:14:53.995281880 (13h11m9.220464927s) [upload=11580]: Upload finished. err: <nil> Duration: 30.002014385s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [...................................X....XX.........XX.XX..XX] duration: 27.565002534s FAIL (2025-08-26 05:15:11 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F393C66397B31, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 27.565024905s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X..........X....X.] duration: 8.321050565s remote: "tcp:136.0.77.2:443" response headers: X-Request-Id=VcCcTMkH9av Content-Length=423 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=185F392FFCED1105 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:14:18 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:14:54 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F392FFCED1105</RequestId><HostId></HostId></Error>" Duration: 8.321070272s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................................X........XX.X.X.X] duration: 589.25639ms FAIL (2025-08-26 05:14:48 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 0704832 (13h11m2.905887889s) [upload=11582]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:14:47.902776638 (13h11m3.127959686s) [seg_pos={0 0}, upload=11582]: Begin upload segment... Aug 26 05:14:47.902820661 (13h11m3.128003709s) [seg_pos={0 0}, upload=11582]: Done begin upload segment. Aug 26 05:14:47.902849054 (13h11m3.128032111s) [noise=true , address=45.91.116.11:4146 , piece_id=HIWPDZR3CTF22YZCYH73QMJFSGET7QTBHRMNOLVMTSU6RJXSWTFA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.902873490 (13h11m3.128056537s) [noise=true , address=77.90.31.14:4694 , piece_id=O6K4VOHGODJGHAKXAGEKEAXU7ZXHIEVTE2S6KBKWSUJVZKACUWNQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903189514 (13h11m3.128372571s) [noise=true , address=104.36.82.13:4214 , piece_id=MBC7UIGS4Y7LUYQKQTG33CMKX7O25ANAV73ONFHCWKX7MLDIXOVA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903342101 (13h11m3.128525148s) [noise=true , address=45.91.116.14:4203 , piece_id=C4YPVPOP56QR7P6RLTPZYXDBOJGNVSI7QS52DQRKESU6MBUEXN2A, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903413926 (13h11m3.128596973s) [noise=true , address=104.36.82.50:4030 , piece_id=FIQJULHAAWQQXVV3BTH5XIIYNUIVLOPXVEEOJWETTY6KBWKCX2WQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903641763 (13h11m3.12882482s) [noise=true , address=79.127.205.246:4268 , piece_id=WMRJ2R3PWW4PEJKMI4WO6RDGV24VBMFDJMNA5ST4XXWNG3EZGYYA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.903697358 (13h11m3.128880406s) [noise=true , address=104.36.82.44:4873 , piece_id=CVFBS5TPDC7DSBY2MGZH4EL5DO3PLJDE2GJCZ2TL6JV7ETU2E34A, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.904382555 (13h11m3.129565612s) [noise=true , address=77.90.31.19:4855 , piece_id=V36EQSYCAE3O2GFK4KNVLKSYOJADKRIOMRBVSOHRI3RNL3AFEYNA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.904391010 (13h11m3.129574068s) [noise=true , address=104.36.82.6:4891 , piece_id=HBARHJPGMLXHPEEA27KEMNMJVRFWTWR74YQL5XTO7KYRZ266VJXA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.904787035 (13h11m3.129970082s) [noise=true , address=104.36.82.60:4354 , piece_id=HFCYUWFHQ6AJA7REULM6KRFPMDFBBKRRJ22VQXSVTM5S4X3BX3PA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.905089312 (13h11m3.130272359s) [noise=true , address=104.36.82.49:4987 , piece_id=ECBZZ23GJA5TWTPB2X7GZBFXIY5VL5FJR77KBF64QCO5BEVUALCQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.905389606 (13h11m3.130572683s) [noise=true , address=104.36.82.10:4085 , piece_id=AZBDQYQUON4UVMJ3F4ULQXTNBB2OSWFFC72KU3MA2J7RGBJZELWA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.905880078 (13h11m3.131063125s) [noise=true , address=79.127.201.215:4349 , piece_id=BU6KUUSE77HTCME4DFHPPMADIFAFHI43PL2JPDEHZHZEYJMRXR4Q, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.906031132 (13h11m3.131214179s) [noise=true , address=45.91.116.7:4030 , piece_id=JX6NWRZODJ3PEDOU43U5KQXHYBFYBPYJNDISY2T3DLK5NYZ5I7GA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.906575734 (13h11m3.131758791s) [noise=true , address=79.127.205.245:4244 , piece_id=G25ULKCGLHJCT7QCOP6ZSPXA7CHOS3KNIRZMRQT6NWRZPVRTCZCQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.906838247 (13h11m3.132021314s) [noise=true , address=45.91.116.6:4992 , piece_id=6G7PY5YY32FTZBPRVLKSHSUH7KNYRIGHAFOPT7AVBCBM7HYH3MVQ, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.907199135 (13h11m3.132382192s) [noise=true , address=45.91.116.13:4169 , piece_id=7ND7C76WDTB77MRO3RPSQDGQFOK7H5WOBSD7RCVI7OUA5H4HSSXA, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.910863127 (13h11m3.136046194s) [noise=true , address=104.36.82.23:4456 , piece_id=NOIRLI7AVEW4P3BRPJI27WOSFOO2WP2E6RDPFWWGHEWUKZ7HTF7Q, seg_pos={0 0}, upload=11582]: Uploading piece... Aug 26 05:14:47.996930147 (13h11m3.222113204s) [upload=11582]: Next returned nil segment Aug 26 05:14:47.996953811 (13h11m3.222136868s) [upload=11582]: Waiting for error group managing segments... Aug 26 05:14:48.004359973 (13h11m3.229543019s) [noise=true , address=45.91.116.7:4030 , piece_id=JX6NWRZODJ3PEDOU43U5KQXHYBFYBPYJNDISY2T3DLK5NYZ5I7GA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.005508650 (13h11m3.230691697s) [noise=true , address=45.91.116.11:4146 , piece_id=HIWPDZR3CTF22YZCYH73QMJFSGET7QTBHRMNOLVMTSU6RJXSWTFA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.005522095 (13h11m3.230705142s) [noise=true , address=45.91.116.14:4203 , piece_id=C4YPVPOP56QR7P6RLTPZYXDBOJGNVSI7QS52DQRKESU6MBUEXN2A, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.006008057 (13h11m3.231191105s) [noise=true , address=45.91.116.13:4169 , piece_id=7ND7C76WDTB77MRO3RPSQDGQFOK7H5WOBSD7RCVI7OUA5H4HSSXA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.008257333 (13h11m3.23344037s) [noise=true , address=104.36.82.49:4987 , piece_id=ECBZZ23GJA5TWTPB2X7GZBFXIY5VL5FJR77KBF64QCO5BEVUALCQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.008662254 (13h11m3.233845311s) [noise=true , address=104.36.82.44:4873 , piece_id=CVFBS5TPDC7DSBY2MGZH4EL5DO3PLJDE2GJCZ2TL6JV7ETU2E34A, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.009403165 (13h11m3.234586212s) [noise=true , address=77.90.31.14:4694 , piece_id=O6K4VOHGODJGHAKXAGEKEAXU7ZXHIEVTE2S6KBKWSUJVZKACUWNQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.009804760 (13h11m3.234987806s) [noise=true , address=77.90.31.19:4855 , piece_id=V36EQSYCAE3O2GFK4KNVLKSYOJADKRIOMRBVSOHRI3RNL3AFEYNA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.011645477 (13h11m3.236828524s) [noise=true , address=104.36.82.13:4214 , piece_id=MBC7UIGS4Y7LUYQKQTG33CMKX7O25ANAV73ONFHCWKX7MLDIXOVA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.016428189 (13h11m3.241611427s) [noise=true , address=104.36.82.50:4030 , piece_id=FIQJULHAAWQQXVV3BTH5XIIYNUIVLOPXVEEOJWETTY6KBWKCX2WQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.016598520 (13h11m3.241781567s) [noise=true , address=104.36.82.10:4085 , piece_id=AZBDQYQUON4UVMJ3F4ULQXTNBB2OSWFFC72KU3MA2J7RGBJZELWA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.017570516 (13h11m3.242753553s) [noise=true , address=104.36.82.6:4891 , piece_id=HBARHJPGMLXHPEEA27KEMNMJVRFWTWR74YQL5XTO7KYRZ266VJXA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.030796789 (13h11m3.255979836s) [noise=true , address=104.36.82.60:4354 , piece_id=HFCYUWFHQ6AJA7REULM6KRFPMDFBBKRRJ22VQXSVTM5S4X3BX3PA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.089222399 (13h11m3.314405456s) [noise=true , address=104.36.82.23:4456 , piece_id=NOIRLI7AVEW4P3BRPJI27WOSFOO2WP2E6RDPFWWGHEWUKZ7HTF7Q, seg_pos={0 0}, upload=11582]: Done uploading piece. err: <nil> Aug 26 05:14:48.089247306 (13h11m3.314430354s) [seg_pos={0 0}, upload=11582]: Segment reached optimal threshold of 14 pieces. Aug 26 05:14:48.089472019 (13h11m3.314655076s) [noise=true , address=45.91.116.6:4992 , piece_id=6G7PY5YY32FTZBPRVLKSHSUH7KNYRIGHAFOPT7AVBCBM7HYH3MVQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ai9G3ULmwAy4zkKPUonML5fgxZGvJJgmqQeGfYHWqUKpehgtx): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:48.089527372 (13h11m3.314710429s) [noise=true , address=79.127.201.215:4349 , piece_id=BU6KUUSE77HTCME4DFHPPMADIFAFHI43PL2JPDEHZHZEYJMRXR4Q, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: write tcp 5.161.89.252:45582->79.127.201.215:4349: use of closed network connection; write tcp 5.161.89.252:45582->79.127.201.215:4349: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45582->79.127.201.215:4349: use of closed network connection Aug 26 05:14:48.089730384 (13h11m3.314913441s) [noise=true , address=79.127.205.246:4268 , piece_id=WMRJ2R3PWW4PEJKMI4WO6RDGV24VBMFDJMNA5ST4XXWNG3EZGYYA, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k72iHWdcnM2JKs7VPTQFQwmpQ835HmAP8KpXxjMbEy5QxfzG): protocol: read tcp 5.161.89.252:10486->79.127.205.246:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:14:48.089765280 (13h11m3.314948327s) [noise=true , address=79.127.205.245:4244 , piece_id=G25ULKCGLHJCT7QCOP6ZSPXA7CHOS3KNIRZMRQT6NWRZPVRTCZCQ, seg_pos={0 0}, upload=11582]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JFy3MxMFZYnjxJmujGpmcTdpVz6qLUMKf1aB3eYW236kCYgng): protocol: read tcp 5.161.89.252:25038->79.127.205.245:4244: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:14:48.089773956 (13h11m3.314957003s) [seg_pos={0 0}, upload=11582]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:14:48.089836283 (13h11m3.31501934s) [upload=11582]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:14:48.089842094 (13h11m3.315025141s) [upload=11582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:48.256048660 (13h11m3.481231707s) [upload=11582]: Done uploading segments... Aug 26 05:14:48.256071674 (13h11m3.481254731s) [upload=11582]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 589.274985ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [...............................X.....XX..........XX....X..X.] duration: 20.202075787s FAIL (2025-08-26 05:14:46 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 20.2021038s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................X.....X...........X.X.XX.XXX] duration: 30.016686167s FAIL (2025-08-26 05:14:41 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:14:11.710295170 (13h10m26.935478227s) [upload=11572]: Starting upload Aug 26 05:14:11.710372595 (13h10m26.935555652s) [upload=11572]: Done starting upload Aug 26 05:14:11.710434711 (13h10m26.935617768s) [upload=11572]: Uploading segments... Aug 26 05:14:11.710634056 (13h10m26.935817113s) [upload=11572]: Got next segment. Inline: false Aug 26 05:14:11.710666717 (13h10m26.935849765s) [upload=11572]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:14:11.710672097 (13h10m26.935855154s) [upload=11572]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:14:15.731290735 (13h10m30.956473782s) [seg_pos={0 0}, upload=11572]: Begin upload segment... Aug 26 05:14:15.731324438 (13h10m30.956507485s) [seg_pos={0 0}, upload=11572]: Done begin upload segment. Aug 26 05:14:15.731340218 (13h10m30.956523265s) [noise=true , address=45.91.116.13:4153 , piece_id=CQ5KH4MGHU2YBZ3Q2TNME262PTB7J22RMQ45BCYJM6NHK32VZABQ, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.731393077 (13h10m30.956576124s) [noise=true , address=104.36.82.13:4185 , piece_id=5744V7QEFA5UJ2XZXA2FGTCETU7JWOD3VLLN7ZF7EJKGGB672EAA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.731651172 (13h10m30.956834219s) [noise=true , address=77.90.31.10:4520 , piece_id=IXSXI3D7HRPFYFL4OION7VAIXBZIOW7DXOGQAM5BXZPMT3ZOHBGA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.731756179 (13h10m30.956939226s) [noise=true , address=45.91.116.14:4189 , piece_id=DX4NECIUIKZL7FYCSUKUYGVS62LI76TIZYAFGVWKKHOQEQLACADA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.731925948 (13h10m30.957108995s) [noise=true , address=104.36.82.7:4910 , piece_id=RCCV4Q3ZIV6WXFZDCOPTSCU32Y4G7PWDQCJLVRTQMAEG6CFW2HCQ, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732070410 (13h10m30.957253467s) [noise=true , address=45.91.116.10:4081 , piece_id=CNQT53Y52KQT2VZZIZCUA2U7ZJWRITGO7ANEPIGVSPBBQUP55HOQ, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732164797 (13h10m30.957347873s) [noise=true , address=45.91.116.8:4048 , piece_id=YLH2KKHP5LYEVPQFVY6LE35RSGBASR75TS4ASTIQPVZZWC3PJGVA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732320750 (13h10m30.957503807s) [noise=true , address=77.90.31.19:4841 , piece_id=J2IUIBB5AFBHMFN2FT6VKB4FISQAPW5B25QNJATOGB2275PP3TVA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732441656 (13h10m30.957624704s) [noise=true , address=104.36.82.12:4151 , piece_id=FDYCIZ62VOAGNXXZNGRAQF43CHYHO7LXCRLSWMTELRDXUSTO6X5Q, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732574396 (13h10m30.957757453s) [noise=true , address=77.90.31.19:4842 , piece_id=TR5VW5ZNNBA5AFHPPDCI23JO5Q64YIY4HYVZPFYTSYCTYLSDFRVA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732681306 (13h10m30.957864374s) [noise=true , address=104.36.82.14:4222 , piece_id=T65I2F3ZPJ6GEYCL6M3JK3RYD6UP5A2LPEP7LNXYQ5JWYSAEAT7Q, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732797495 (13h10m30.957980542s) [noise=true , address=45.91.116.8:4047 , piece_id=2PBCHK7GUM4TNUWVIEOKLKDFDSAPUJ6E3TC24SIMRQOZBENE5SOQ, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.732941285 (13h10m30.958124332s) [noise=true , address=45.91.116.12:4227 , piece_id=YVD3KQYQM42UPCK6PNKW53RMRSYBS5HEEEXGDKK235KLWG7P22UA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.733048376 (13h10m30.958231433s) [noise=true , address=77.90.31.15:4698 , piece_id=SWIVEJO2NV4FYRGMJJPD6E2WFK4AHG3KJHG22ZUZ77GJU27I4ACA, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.733175064 (13h10m30.958358111s) [noise=true , address=104.36.82.5:4838 , piece_id=NL5CORQKDOU35BSB3KIIGSW3Z5SC6WAPAX6ZA6DXAGHQSL6XE6FQ, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.733443107 (13h10m30.958626154s) [noise=true , address=45.91.116.10:4082 , piece_id=YT4F6JM3W5LPRWW4S7YTVUAZXSGC6U5W3KJCDZI3ZBNCCMCJ727Q, seg_pos={0 0}, upload=11572]: Uploading piece... Aug 26 05:14:15.841629871 (13h10m31.066812908s) [upload=11572]: Next returned nil segment Aug 26 05:14:15.841642575 (13h10m31.066825622s) [upload=11572]: Waiting for error group managing segments... Aug 26 05:14:15.850026039 (13h10m31.075209086s) [noise=true , address=45.91.116.8:4047 , piece_id=2PBCHK7GUM4TNUWVIEOKLKDFDSAPUJ6E3TC24SIMRQOZBENE5SOQ, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.851146393 (13h10m31.07632944s) [noise=true , address=45.91.116.14:4189 , piece_id=DX4NECIUIKZL7FYCSUKUYGVS62LI76TIZYAFGVWKKHOQEQLACADA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.851360205 (13h10m31.076543282s) [noise=true , address=45.91.116.12:4227 , piece_id=YVD3KQYQM42UPCK6PNKW53RMRSYBS5HEEEXGDKK235KLWG7P22UA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.851595057 (13h10m31.076778104s) [noise=true , address=45.91.116.10:4082 , piece_id=YT4F6JM3W5LPRWW4S7YTVUAZXSGC6U5W3KJCDZI3ZBNCCMCJ727Q, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.852014765 (13h10m31.077197812s) [noise=true , address=77.90.31.15:4698 , piece_id=SWIVEJO2NV4FYRGMJJPD6E2WFK4AHG3KJHG22ZUZ77GJU27I4ACA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.852376164 (13h10m31.077559211s) [noise=true , address=45.91.116.10:4081 , piece_id=CNQT53Y52KQT2VZZIZCUA2U7ZJWRITGO7ANEPIGVSPBBQUP55HOQ, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.852396281 (13h10m31.077579358s) [noise=true , address=45.91.116.8:4048 , piece_id=YLH2KKHP5LYEVPQFVY6LE35RSGBASR75TS4ASTIQPVZZWC3PJGVA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.853126793 (13h10m31.07830985s) [noise=true , address=77.90.31.19:4842 , piece_id=TR5VW5ZNNBA5AFHPPDCI23JO5Q64YIY4HYVZPFYTSYCTYLSDFRVA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.853336306 (13h10m31.078519343s) [noise=true , address=77.90.31.10:4520 , piece_id=IXSXI3D7HRPFYFL4OION7VAIXBZIOW7DXOGQAM5BXZPMT3ZOHBGA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.855597744 (13h10m31.080780791s) [noise=true , address=45.91.116.13:4153 , piece_id=CQ5KH4MGHU2YBZ3Q2TNME262PTB7J22RMQ45BCYJM6NHK32VZABQ, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.855754478 (13h10m31.080937536s) [noise=true , address=77.90.31.19:4841 , piece_id=J2IUIBB5AFBHMFN2FT6VKB4FISQAPW5B25QNJATOGB2275PP3TVA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.856942330 (13h10m31.082125376s) [noise=true , address=104.36.82.7:4910 , piece_id=RCCV4Q3ZIV6WXFZDCOPTSCU32Y4G7PWDQCJLVRTQMAEG6CFW2HCQ, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.858142042 (13h10m31.083325109s) [noise=true , address=104.36.82.13:4185 , piece_id=5744V7QEFA5UJ2XZXA2FGTCETU7JWOD3VLLN7ZF7EJKGGB672EAA, seg_pos={0 0}, upload=11572]: Done uploading piece. err: <nil> Aug 26 05:14:15.858155908 (13h10m31.083338965s) [seg_pos={0 0}, upload=11572]: Segment reached optimal threshold of 13 pieces. Aug 26 05:14:15.858285191 (13h10m31.083468248s) [noise=true , address=104.36.82.5:4838 , piece_id=NL5CORQKDOU35BSB3KIIGSW3Z5SC6WAPAX6ZA6DXAGHQSL6XE6FQ, seg_pos={0 0}, upload=11572]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:15.858389256 (13h10m31.083572323s) [noise=true , address=104.36.82.14:4222 , piece_id=T65I2F3ZPJ6GEYCL6M3JK3RYD6UP5A2LPEP7LNXYQ5JWYSAEAT7Q, seg_pos={0 0}, upload=11572]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:15.858479716 (13h10m31.083662783s) [noise=true , address=104.36.82.12:4151 , piece_id=FDYCIZ62VOAGNXXZNGRAQF43CHYHO7LXCRLSWMTELRDXUSTO6X5Q, seg_pos={0 0}, upload=11572]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB): protocol: expected piece hash; context canceled; EOF Aug 26 05:14:15.858499343 (13h10m31.08368241s) [seg_pos={0 0}, upload=11572]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:14:15.858580175 (13h10m31.083763242s) [upload=11572]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:14:15.858587709 (13h10m31.083770776s) [upload=11572]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:41.715092247 (13h10m56.940275324s) [upload=11572]: Done uploading segments... Aug 26 05:14:41.715120701 (13h10m56.940303777s) [upload=11572]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.016707145s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [...............................XX.......X..........XX.X.X.XX] duration: 12.545651084s FAIL (2025-08-26 05:14:41 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:14:29.029036501 (13h10m44.254219628s) [upload=11579]: Starting upload Aug 26 05:14:29.029135167 (13h10m44.254318244s) [upload=11579]: Done starting upload Aug 26 05:14:29.029207823 (13h10m44.25439091s) [upload=11579]: Uploading segments... Aug 26 05:14:29.029253299 (13h10m44.254436376s) [upload=11579]: Got next segment. Inline: true Aug 26 05:14:29.029259290 (13h10m44.254442367s) [upload=11579]: Waiting for error group managing segments... Aug 26 05:14:29.029307520 (13h10m44.254490598s) [upload=11579]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:14:29.029313171 (13h10m44.254496248s) [upload=11579]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:14:29.029317299 (13h10m44.254500376s) [upload=11579]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:41.554926683 (13h10m56.78010974s) [upload=11579]: Done uploading segments... Aug 26 05:14:41.554940789 (13h10m56.780123846s) [upload=11579]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 12.545667394s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X.....XX.] duration: 7.896655775s remote: "tcp:45.38.24.2:443" response headers: Date=Tue, 26 Aug 2025 05:13:50 GMT Content-Length=433 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3929769960CB X-Request-Id=W9w6VLVezcu X-Xss-Protection=1; mode=block target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:14:40 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3929769960CB</RequestId><HostId></HostId></Error>" Duration: 7.896679379s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................X.X..X.] duration: 20.945412312s FAIL (2025-08-26 05:14:29 UTC) operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3927EAB52717, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 20.945435126s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................................X........XX.X.X.] duration: 21.452919171s FAIL (2025-08-26 05:14:28 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== estItem_ObjectBegin Aug 26 05:13:41.276772745 (13h9m56.501955821s) [upload=11565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:13:41.362115846 (13h9m56.587298903s) [seg_pos={0 0}, upload=11565]: Begin upload segment... Aug 26 05:13:41.362150060 (13h9m56.587333117s) [seg_pos={0 0}, upload=11565]: Done begin upload segment. Aug 26 05:13:41.362170117 (13h9m56.587353174s) [noise=true , address=104.36.82.13:4203 , piece_id=EZS5HYDFGSVI6LMISBGO3N4QCAM7GWZUCTLIL6JXIJAIEPLW62FA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362204692 (13h9m56.587387749s) [noise=true , address=79.127.205.246:4254 , piece_id=EW3ACHT6SG4VHW6DSZ2TBI55Q75G6VRNH4X7BFQQCEV6WMVYYW5Q, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362487334 (13h9m56.587670381s) [noise=true , address=79.127.201.214:4299 , piece_id=XA5YQKO3E6P537ANACWNTISC3KM7XTNP7DWL2JYHHUIYQOX4UWPA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362517300 (13h9m56.587700357s) [noise=true , address=45.91.116.10:4096 , piece_id=5UFXZTBHAHGO66MIKNHEYQDHR6TU3FPSHS4QAEWCWLKB7KS5E3VQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362731593 (13h9m56.58791464s) [noise=true , address=77.90.31.13:4640 , piece_id=YL5GO7EZG6NEUAV6W7VNLUVMC6COHESWJAL7SOKSMHNDDZPGVSXQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362794511 (13h9m56.587977558s) [noise=true , address=193.149.164.143:4694 , piece_id=JQ3CZ3LRSCMD3LTT7P3PVMHDMWMNRBYMZDEDYXXBEF2WZEVIT6QA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362979227 (13h9m56.588162284s) [noise=true , address=45.91.116.5:4310 , piece_id=JWV5PSICVRAUEHE2ZMPSLZQHF6YW4UFPOOGUX2RVKCY7CFSARXBQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363058225 (13h9m56.588241282s) [noise=true , address=64.49.13.140:4508 , piece_id=G7Y7QOFOOMRSVVELUCHKK4IGX6NPPKPZPKHRN2G7DSWLYVETSKJQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363243363 (13h9m56.58842642s) [noise=true , address=104.36.82.60:4368 , piece_id=QDAYLFDR53XS5JW7UQRTA7QVCRIUDD2DVQXJEFPHH6PXTXFHZ7XQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363336929 (13h9m56.588519986s) [noise=true , address=79.127.205.234:4620 , piece_id=QLOKDNIIFRRTWPRBZBSSYY5LBZ5OE2JAORZBK5R464CMFDUNIZQQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363506016 (13h9m56.588689063s) [noise=true , address=104.36.82.44:4899 , piece_id=5GUG5G27NPKBMMP6UPANTMOVHGWIZOQCFQIMNUAXZNLADOQOHKNQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363610032 (13h9m56.588793089s) [noise=true , address=77.90.31.21:4881 , piece_id=6UQY5WOUIA2J6U75DBATVIMBTKF6VKOG5JHFMICL6YNF6MGFFZJQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363767528 (13h9m56.588950585s) [noise=true , address=77.90.31.19:4852 , piece_id=I22RF3HMIKEKYKHPRXRYN6KW3OMFDN646KC4HR24DQL23SOJRBOQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363943709 (13h9m56.589126766s) [noise=true , address=79.127.201.215:4343 , piece_id=NGLZCPZSWNVRU2EOKURK7EYIBBQ5USTN4EIB34AXJWPWAHDRGBEQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364038667 (13h9m56.589221714s) [noise=true , address=104.36.82.5:4869 , piece_id=QCCC7642LGPS5RXY7DAHDGPZTNGLASMHXZQP7KSVLYCOOXCXNLFQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364186695 (13h9m56.589369742s) [noise=true , address=45.91.116.12:4251 , piece_id=LATWDNZP36TIMTDXPCUZF7BOMCRPIS5IZ4PPXZFAPE4PLRHTY3ZQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364313573 (13h9m56.589496621s) [noise=true , address=104.36.82.15:4292 , piece_id=A7A56VDLWRDCIU72YQUUPUFZ7AHSYTZQK7SSUQOG6ZPSLQARI7VA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364600521 (13h9m56.589783579s) [noise=true , address=79.127.219.47:4057 , piece_id=L4YHP36262XOVY2LYVY7XLEO426GL4MUQOSBUWMPER5OJSZELEIA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.449661793 (13h9m56.674844841s) [upload=11565]: Next returned nil segment Aug 26 05:13:41.449672524 (13h9m56.674855571s) [upload=11565]: Waiting for error group managing segments... Aug 26 05:13:41.456051655 (13h9m56.681234712s) [noise=true , address=79.127.219.47:4057 , piece_id=L4YHP36262XOVY2LYVY7XLEO426GL4MUQOSBUWMPER5OJSZELEIA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.457086718 (13h9m56.682269775s) [noise=true , address=45.91.116.10:4096 , piece_id=5UFXZTBHAHGO66MIKNHEYQDHR6TU3FPSHS4QAEWCWLKB7KS5E3VQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.457243393 (13h9m56.68242644s) [noise=true , address=45.91.116.12:4251 , piece_id=LATWDNZP36TIMTDXPCUZF7BOMCRPIS5IZ4PPXZFAPE4PLRHTY3ZQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.457840053 (13h9m56.68302311s) [noise=true , address=45.91.116.5:4310 , piece_id=JWV5PSICVRAUEHE2ZMPSLZQHF6YW4UFPOOGUX2RVKCY7CFSARXBQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.459597013 (13h9m56.68478007s) [noise=true , address=77.90.31.21:4881 , piece_id=6UQY5WOUIA2J6U75DBATVIMBTKF6VKOG5JHFMICL6YNF6MGFFZJQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.460289213 (13h9m56.685472261s) [noise=true , address=77.90.31.13:4640 , piece_id=YL5GO7EZG6NEUAV6W7VNLUVMC6COHESWJAL7SOKSMHNDDZPGVSXQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.460768433 (13h9m56.685951481s) [noise=true , address=77.90.31.19:4852 , piece_id=I22RF3HMIKEKYKHPRXRYN6KW3OMFDN646KC4HR24DQL23SOJRBOQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.461901491 (13h9m56.687084538s) [noise=true , address=104.36.82.13:4203 , piece_id=EZS5HYDFGSVI6LMISBGO3N4QCAM7GWZUCTLIL6JXIJAIEPLW62FA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.461909817 (13h9m56.687092874s) [noise=true , address=104.36.82.5:4869 , piece_id=QCCC7642LGPS5RXY7DAHDGPZTNGLASMHXZQP7KSVLYCOOXCXNLFQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.462644336 (13h9m56.687827383s) [noise=true , address=104.36.82.44:4899 , piece_id=5GUG5G27NPKBMMP6UPANTMOVHGWIZOQCFQIMNUAXZNLADOQOHKNQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.469009209 (13h9m56.694192267s) [noise=true , address=104.36.82.15:4292 , piece_id=A7A56VDLWRDCIU72YQUUPUFZ7AHSYTZQK7SSUQOG6ZPSLQARI7VA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.485238089 (13h9m56.710421136s) [noise=true , address=104.36.82.60:4368 , piece_id=QDAYLFDR53XS5JW7UQRTA7QVCRIUDD2DVQXJEFPHH6PXTXFHZ7XQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.606276829 (13h9m56.831459876s) [noise=true , address=79.127.201.214:4299 , piece_id=XA5YQKO3E6P537ANACWNTISC3KM7XTNP7DWL2JYHHUIYQOX4UWPA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.622092552 (13h9m56.84727563s) [noise=true , address=79.127.201.215:4343 , piece_id=NGLZCPZSWNVRU2EOKURK7EYIBBQ5USTN4EIB34AXJWPWAHDRGBEQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.622130043 (13h9m56.84731313s) [seg_pos={0 0}, upload=11565]: Segment reached optimal threshold of 14 pieces. Aug 26 05:13:41.622992032 (13h9m56.84817511s) [noise=true , address=64.49.13.140:4508 , piece_id=G7Y7QOFOOMRSVVELUCHKK4IGX6NPPKPZPKHRN2G7DSWLYVETSKJQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1diUU2skKzXPjYy3Mu3iDDjnG5W8Z2cxCRMxDCH9cQGg2TdEiR): protocol: read tcp 5.161.89.252:56940->64.49.13.140:4508: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623005398 (13h9m56.848188474s) [noise=true , address=79.127.205.234:4620 , piece_id=QLOKDNIIFRRTWPRBZBSSYY5LBZ5OE2JAORZBK5R464CMFDUNIZQQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CvVNHcgV57c7VTpV97A8kXNuV3grxknxgNL9kJFZo9aKqo6L4): protocol: read tcp 5.161.89.252:38838->79.127.205.234:4620: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623062946 (13h9m56.848246022s) [noise=true , address=193.149.164.143:4694 , piece_id=JQ3CZ3LRSCMD3LTT7P3PVMHDMWMNRBYMZDEDYXXBEF2WZEVIT6QA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ATtVD3XyFxzEqwQpSF4t1MWygqEa4Y1CkNj1s43eud8UtuSmx): protocol: read tcp 5.161.89.252:46402->193.149.164.143:4694: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623102279 (13h9m56.848285357s) [noise=true , address=79.127.205.246:4254 , piece_id=EW3ACHT6SG4VHW6DSZ2TBI55Q75G6VRNH4X7BFQQCEV6WMVYYW5Q, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zrPqM7xhRpeXnvSRp2dssRupk6zgCK7KTJJFxoVQvXGmF5445): protocol: read tcp 5.161.89.252:53940->79.127.205.246:4254: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623133287 (13h9m56.848316364s) [seg_pos={0 0}, upload=11565]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:13:41.623206976 (13h9m56.848390062s) [upload=11565]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:13:41.623215051 (13h9m56.848398138s) [upload=11565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:13:41.947765180 (13h9m57.172948258s) [upload=11565]: Done uploading segments... Aug 26 05:13:41.947795217 (13h9m57.172978294s) [upload=11565]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 21.452935981s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [................................X.....XX..........XX....X..X] duration: 30.001144322s FAIL (2025-08-26 05:14:25 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.001167275s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...............................XX.......X..X.......XXXXXXXX.] duration: 21.551645344s FAIL (2025-08-26 05:14:23 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 21.551658337s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [....................................X....XX.........XX.XX..X] duration: 30.000568051s FAIL (2025-08-26 05:14:23 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000589541s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................X..................XXX....XX] duration: 2.021310535s FAIL (2025-08-26 05:14:20 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F3930AA2D7D1C, HostID: , api error SlowDown: Please reduce your request rate. Duration: 2.021343528s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X..........X....X] duration: 253.890828ms remote: "tcp:136.0.77.2:443" response headers: X-Request-Id=VcCcTMkH9av Content-Length=423 Content-Type=application/xml Vary=Origin X-Amz-Request-Id=185F392FFCED1105 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:14:18 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:14:18 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F392FFCED1105</RequestId><HostId></HostId></Error>" Duration: 253.914713ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................X.....X...........X.X.XX.XX] duration: 30.002105127s FAIL (2025-08-26 05:14:11 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:13:41.658059552 (13h9m56.883242609s) [upload=11566]: Starting upload Aug 26 05:13:41.658137899 (13h9m56.883320956s) [upload=11566]: Done starting upload Aug 26 05:13:41.658276850 (13h9m56.883459927s) [upload=11566]: Uploading segments... Aug 26 05:13:41.658449815 (13h9m56.883632891s) [upload=11566]: Got next segment. Inline: false Aug 26 05:13:41.658503134 (13h9m56.883686212s) [upload=11566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:41.658512532 (13h9m56.88369562s) [upload=11566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:13:45.680563197 (13h10m0.905746285s) [seg_pos={0 0}, upload=11566]: Begin upload segment... Aug 26 05:13:45.680623491 (13h10m0.905806568s) [seg_pos={0 0}, upload=11566]: Done begin upload segment. Aug 26 05:13:45.680658116 (13h10m0.905841193s) [noise=true , address=77.90.31.18:4805 , piece_id=ERD2D7KW7HP3AQMMB34BNOA7T3G5BO22JOSS54SJHLDDID6SIP6Q, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.680694363 (13h10m0.90587742s) [noise=true , address=77.90.31.11:4555 , piece_id=YD7G3ZCPSB5JOUUOEVEBI3GY4CD6DGZJBFWVJNAGZRGPMJIQSMSA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.681201216 (13h10m0.906384293s) [noise=true , address=45.91.116.12:4225 , piece_id=N2BWZC4XWWVW3KNLSH53V5XGYUF5JJ7MEBHC6SZEFUBWIEJXTYPQ, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.681267812 (13h10m0.906450869s) [noise=true , address=45.91.116.5:4297 , piece_id=6WYDSDMBAB23T5ZBM3H2RL5KJIZ6YGAKKISVWUK7YPGI657MPYHA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.681578936 (13h10m0.906762023s) [noise=true , address=45.91.116.8:4048 , piece_id=NMLCAIEQ7M6YKZ43ZHKHQFY42RORG2TF6BN3KJA76JLZFAQ6J46A, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.681695404 (13h10m0.906878461s) [noise=true , address=45.91.116.7:4011 , piece_id=A4FOT5QFHUDX5IAWYLBZWN42MEWPGIDXG3HVNLRLSOGUCOWD2GAQ, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.681969910 (13h10m0.907152987s) [noise=true , address=45.91.116.4:4938 , piece_id=D2ORR4WL3FROBHYGK7NMMJEQIRW6HBQMNAPABWD6PF75VE26HZCQ, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.682009945 (13h10m0.907192992s) [noise=true , address=104.36.82.9:4006 , piece_id=K5EBYFLMGE4AFKUPPAXOV3JRLEWMPKDGPUUI4TYJHZVBIDQTDJBQ, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.682345045 (13h10m0.907528121s) [noise=true , address=77.90.31.10:4517 , piece_id=QZGKS5ODT7KE3TWRQQMHYJ5CCYRZS7R7EOU75CC3A3UCP2XK5MRQ, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.682418543 (13h10m0.9076016s) [noise=true , address=77.90.31.19:4843 , piece_id=STKUKRDEGSEUV32TLSBYJBHBZJU4AU3PICJYZPNK3LDKMT7YBBMA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.682708768 (13h10m0.907891845s) [noise=true , address=104.36.82.11:4114 , piece_id=O7DACXV2AXGZ4KIOVQGCLV342YFYJWXLNMDHQ52CR5U74HG7EWKQ, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.682784700 (13h10m0.907967747s) [noise=true , address=77.90.31.12:4589 , piece_id=UDTQ5GZJIVA3CTWN67RCGPY6AZWLQVXKC4PL6UEK3YFLYQ67WLHA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.683101155 (13h10m0.908284232s) [noise=true , address=77.90.31.16:4733 , piece_id=PI4P6AH5QJERGBDJIOWOOLG6F2W5IH7KAZV4KLNWRXCZGTLYHRZA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.683170855 (13h10m0.908353912s) [noise=true , address=77.90.31.16:4734 , piece_id=NSU7PQDNRWXRROPEDUYWAJVNJUF4GFAQWK4DG7A5ZZZLGMQOGKDA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.683445582 (13h10m0.908628639s) [noise=true , address=45.91.116.14:4190 , piece_id=D4SIEZI3ZQK3GYKDCHC2UNBKNM7I23G4GB2YQKU3NJLIL437KBBA, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.683458526 (13h10m0.908641603s) [noise=true , address=104.36.82.2:4732 , piece_id=UQZUGUKUIH5WLZP4ALMDMQKZJGKIOOQHEOR46YUTAEF5ZIY5JC5A, seg_pos={0 0}, upload=11566]: Uploading piece... Aug 26 05:13:45.783485414 (13h10m1.008668491s) [upload=11566]: Next returned nil segment Aug 26 05:13:45.783509609 (13h10m1.008692696s) [upload=11566]: Waiting for error group managing segments... Aug 26 05:13:45.792703727 (13h10m1.017886784s) [noise=true , address=45.91.116.5:4297 , piece_id=6WYDSDMBAB23T5ZBM3H2RL5KJIZ6YGAKKISVWUK7YPGI657MPYHA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.793139806 (13h10m1.018322863s) [noise=true , address=45.91.116.14:4190 , piece_id=D4SIEZI3ZQK3GYKDCHC2UNBKNM7I23G4GB2YQKU3NJLIL437KBBA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.793374736 (13h10m1.018557793s) [noise=true , address=45.91.116.4:4938 , piece_id=D2ORR4WL3FROBHYGK7NMMJEQIRW6HBQMNAPABWD6PF75VE26HZCQ, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.793739041 (13h10m1.018922087s) [noise=true , address=45.91.116.12:4225 , piece_id=N2BWZC4XWWVW3KNLSH53V5XGYUF5JJ7MEBHC6SZEFUBWIEJXTYPQ, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.793962581 (13h10m1.019145648s) [noise=true , address=45.91.116.7:4011 , piece_id=A4FOT5QFHUDX5IAWYLBZWN42MEWPGIDXG3HVNLRLSOGUCOWD2GAQ, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.794481195 (13h10m1.019664241s) [noise=true , address=77.90.31.12:4589 , piece_id=UDTQ5GZJIVA3CTWN67RCGPY6AZWLQVXKC4PL6UEK3YFLYQ67WLHA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.794932763 (13h10m1.02011582s) [noise=true , address=77.90.31.10:4517 , piece_id=QZGKS5ODT7KE3TWRQQMHYJ5CCYRZS7R7EOU75CC3A3UCP2XK5MRQ, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.795374342 (13h10m1.020557399s) [noise=true , address=45.91.116.8:4048 , piece_id=NMLCAIEQ7M6YKZ43ZHKHQFY42RORG2TF6BN3KJA76JLZFAQ6J46A, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.795749326 (13h10m1.020932383s) [noise=true , address=77.90.31.16:4733 , piece_id=PI4P6AH5QJERGBDJIOWOOLG6F2W5IH7KAZV4KLNWRXCZGTLYHRZA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.796487663 (13h10m1.02167072s) [noise=true , address=77.90.31.11:4555 , piece_id=YD7G3ZCPSB5JOUUOEVEBI3GY4CD6DGZJBFWVJNAGZRGPMJIQSMSA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.796873287 (13h10m1.022056334s) [noise=true , address=104.36.82.11:4114 , piece_id=O7DACXV2AXGZ4KIOVQGCLV342YFYJWXLNMDHQ52CR5U74HG7EWKQ, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.797805789 (13h10m1.022988866s) [noise=true , address=77.90.31.18:4805 , piece_id=ERD2D7KW7HP3AQMMB34BNOA7T3G5BO22JOSS54SJHLDDID6SIP6Q, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.798475637 (13h10m1.023658715s) [noise=true , address=77.90.31.16:4734 , piece_id=NSU7PQDNRWXRROPEDUYWAJVNJUF4GFAQWK4DG7A5ZZZLGMQOGKDA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: <nil> Aug 26 05:13:45.798500354 (13h10m1.023683441s) [seg_pos={0 0}, upload=11566]: Segment reached optimal threshold of 13 pieces. Aug 26 05:13:45.798632792 (13h10m1.023815869s) [noise=true , address=77.90.31.19:4843 , piece_id=STKUKRDEGSEUV32TLSBYJBHBZJU4AU3PICJYZPNK3LDKMT7YBBMA, seg_pos={0 0}, upload=11566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF Aug 26 05:13:45.798755642 (13h10m1.02393872s) [noise=true , address=104.36.82.9:4006 , piece_id=K5EBYFLMGE4AFKUPPAXOV3JRLEWMPKDGPUUI4TYJHZVBIDQTDJBQ, seg_pos={0 0}, upload=11566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF Aug 26 05:13:45.798865810 (13h10m1.024048887s) [noise=true , address=104.36.82.2:4732 , piece_id=UQZUGUKUIH5WLZP4ALMDMQKZJGKIOOQHEOR46YUTAEF5ZIY5JC5A, seg_pos={0 0}, upload=11566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 26 05:13:45.798880638 (13h10m1.024063715s) [seg_pos={0 0}, upload=11566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:13:45.798997667 (13h10m1.024180754s) [upload=11566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:13:45.799004340 (13h10m1.024187417s) [upload=11566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:13:59.056511865 (13h10m14.281694922s) [upload=11566]: Done uploading segments... Aug 26 05:13:59.056529358 (13h10m14.281712415s) [upload=11566]: Upload finished. err: <nil> Duration: 30.002205026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................X.X..XX] duration: 25.938349607s FAIL (2025-08-26 05:14:09 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:13:43.987058282 (13h9m59.212241339s) [upload=11567]: Starting upload Aug 26 05:13:43.987144814 (13h9m59.212327871s) [upload=11567]: Done starting upload Aug 26 05:13:43.987212292 (13h9m59.212395349s) [upload=11567]: Uploading segments... Aug 26 05:13:43.987257306 (13h9m59.212440363s) [upload=11567]: Got next segment. Inline: true Aug 26 05:13:43.987263478 (13h9m59.212446525s) [upload=11567]: Waiting for error group managing segments... Aug 26 05:13:43.987304054 (13h9m59.212487152s) [upload=11567]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:43.987309574 (13h9m59.212492631s) [upload=11567]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:13:43.987313713 (13h9m59.21249677s) [upload=11567]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:14:09.912697999 (13h10m25.137881087s) [upload=11567]: Done uploading segments... Aug 26 05:14:09.912726052 (13h10m25.137909209s) [upload=11567]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 25.938368753s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 12 out of 60 is failed (threshold=3) [................................XX.......X..X.......XXXXXXXX] duration: 30.00332498s FAIL (2025-08-26 05:14:02 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.003342923s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 3 out of 60 is failed (threshold=6) [.....................................X...............X..X...] duration: 5.140126518s remote: "tcp:136.0.77.2:443" response headers: Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:12:16 GMT X-Request-Id=cG2LXHRGapy Content-Type=text/html; charset=utf-8 target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:13:55 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 5.140145343s edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X.....XX] duration: 288.4919ms remote: "tcp:45.38.24.2:443" response headers: Date=Tue, 26 Aug 2025 05:13:50 GMT Content-Length=433 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3929769960CB X-Request-Id=W9w6VLVezcu X-Xss-Protection=1; mode=block target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:13:50 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3929769960CB</RequestId><HostId></HostId></Error>" Duration: 288.507789ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 8 out of 60 is failed (threshold=3) [................................XX.......X..........XX.X.X.X] duration: 283.058355ms FAIL (2025-08-26 05:13:47 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:13:47.296592218 (13h10m2.521775265s) [upload=11568]: Starting upload Aug 26 05:13:47.296669684 (13h10m2.521852741s) [upload=11568]: Done starting upload Aug 26 05:13:47.296722644 (13h10m2.52190569s) [upload=11568]: Uploading segments... Aug 26 05:13:47.296753912 (13h10m2.521936959s) [upload=11568]: Got next segment. Inline: true Aug 26 05:13:47.296757839 (13h10m2.521940897s) [upload=11568]: Waiting for error group managing segments... Aug 26 05:13:47.296792003 (13h10m2.52197506s) [upload=11568]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:47.296795580 (13h10m2.521978638s) [upload=11568]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:13:47.296798105 (13h10m2.521981162s) [upload=11568]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:13:47.566623316 (13h10m2.791806373s) [upload=11568]: Done uploading segments... Aug 26 05:13:47.566637653 (13h10m2.791820711s) [upload=11568]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 283.077963ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X.X..X] duration: 19.617541305s FAIL (2025-08-26 05:13:43 UTC) operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:13:24.299491164 (13h9m39.52467425s) [upload=11562]: Starting upload Aug 26 05:13:24.299610817 (13h9m39.524793895s) [upload=11562]: Done starting upload Aug 26 05:13:24.299701238 (13h9m39.524884314s) [upload=11562]: Uploading segments... Aug 26 05:13:24.299756973 (13h9m39.52494005s) [upload=11562]: Got next segment. Inline: true Aug 26 05:13:24.299766901 (13h9m39.524949979s) [upload=11562]: Waiting for error group managing segments... Aug 26 05:13:24.299826102 (13h9m39.52500918s) [upload=11562]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:24.299831713 (13h9m39.52501479s) [upload=11562]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:13:24.299838516 (13h9m39.525021603s) [upload=11562]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:13:26.846434812 (13h9m42.071617869s) [upload=11562]: Done uploading segments... Aug 26 05:13:26.846448858 (13h9m42.071631915s) [upload=11562]: Upload finished. err: <nil> Duration:* 19.617561843s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X.X..X] duration: 8.269271646s FAIL (2025-08-26 05:13:43 UTC) operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3927EAB52717, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 8.269290172s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................................X........XX.X.X] duration: 697.398108ms FAIL (2025-08-26 05:13:41 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== estItem_ObjectBegin Aug 26 05:13:41.276772745 (13h9m56.501955821s) [upload=11565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:13:41.362115846 (13h9m56.587298903s) [seg_pos={0 0}, upload=11565]: Begin upload segment... Aug 26 05:13:41.362150060 (13h9m56.587333117s) [seg_pos={0 0}, upload=11565]: Done begin upload segment. Aug 26 05:13:41.362170117 (13h9m56.587353174s) [noise=true , address=104.36.82.13:4203 , piece_id=EZS5HYDFGSVI6LMISBGO3N4QCAM7GWZUCTLIL6JXIJAIEPLW62FA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362204692 (13h9m56.587387749s) [noise=true , address=79.127.205.246:4254 , piece_id=EW3ACHT6SG4VHW6DSZ2TBI55Q75G6VRNH4X7BFQQCEV6WMVYYW5Q, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362487334 (13h9m56.587670381s) [noise=true , address=79.127.201.214:4299 , piece_id=XA5YQKO3E6P537ANACWNTISC3KM7XTNP7DWL2JYHHUIYQOX4UWPA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362517300 (13h9m56.587700357s) [noise=true , address=45.91.116.10:4096 , piece_id=5UFXZTBHAHGO66MIKNHEYQDHR6TU3FPSHS4QAEWCWLKB7KS5E3VQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362731593 (13h9m56.58791464s) [noise=true , address=77.90.31.13:4640 , piece_id=YL5GO7EZG6NEUAV6W7VNLUVMC6COHESWJAL7SOKSMHNDDZPGVSXQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362794511 (13h9m56.587977558s) [noise=true , address=193.149.164.143:4694 , piece_id=JQ3CZ3LRSCMD3LTT7P3PVMHDMWMNRBYMZDEDYXXBEF2WZEVIT6QA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.362979227 (13h9m56.588162284s) [noise=true , address=45.91.116.5:4310 , piece_id=JWV5PSICVRAUEHE2ZMPSLZQHF6YW4UFPOOGUX2RVKCY7CFSARXBQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363058225 (13h9m56.588241282s) [noise=true , address=64.49.13.140:4508 , piece_id=G7Y7QOFOOMRSVVELUCHKK4IGX6NPPKPZPKHRN2G7DSWLYVETSKJQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363243363 (13h9m56.58842642s) [noise=true , address=104.36.82.60:4368 , piece_id=QDAYLFDR53XS5JW7UQRTA7QVCRIUDD2DVQXJEFPHH6PXTXFHZ7XQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363336929 (13h9m56.588519986s) [noise=true , address=79.127.205.234:4620 , piece_id=QLOKDNIIFRRTWPRBZBSSYY5LBZ5OE2JAORZBK5R464CMFDUNIZQQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363506016 (13h9m56.588689063s) [noise=true , address=104.36.82.44:4899 , piece_id=5GUG5G27NPKBMMP6UPANTMOVHGWIZOQCFQIMNUAXZNLADOQOHKNQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363610032 (13h9m56.588793089s) [noise=true , address=77.90.31.21:4881 , piece_id=6UQY5WOUIA2J6U75DBATVIMBTKF6VKOG5JHFMICL6YNF6MGFFZJQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363767528 (13h9m56.588950585s) [noise=true , address=77.90.31.19:4852 , piece_id=I22RF3HMIKEKYKHPRXRYN6KW3OMFDN646KC4HR24DQL23SOJRBOQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.363943709 (13h9m56.589126766s) [noise=true , address=79.127.201.215:4343 , piece_id=NGLZCPZSWNVRU2EOKURK7EYIBBQ5USTN4EIB34AXJWPWAHDRGBEQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364038667 (13h9m56.589221714s) [noise=true , address=104.36.82.5:4869 , piece_id=QCCC7642LGPS5RXY7DAHDGPZTNGLASMHXZQP7KSVLYCOOXCXNLFQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364186695 (13h9m56.589369742s) [noise=true , address=45.91.116.12:4251 , piece_id=LATWDNZP36TIMTDXPCUZF7BOMCRPIS5IZ4PPXZFAPE4PLRHTY3ZQ, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364313573 (13h9m56.589496621s) [noise=true , address=104.36.82.15:4292 , piece_id=A7A56VDLWRDCIU72YQUUPUFZ7AHSYTZQK7SSUQOG6ZPSLQARI7VA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.364600521 (13h9m56.589783579s) [noise=true , address=79.127.219.47:4057 , piece_id=L4YHP36262XOVY2LYVY7XLEO426GL4MUQOSBUWMPER5OJSZELEIA, seg_pos={0 0}, upload=11565]: Uploading piece... Aug 26 05:13:41.449661793 (13h9m56.674844841s) [upload=11565]: Next returned nil segment Aug 26 05:13:41.449672524 (13h9m56.674855571s) [upload=11565]: Waiting for error group managing segments... Aug 26 05:13:41.456051655 (13h9m56.681234712s) [noise=true , address=79.127.219.47:4057 , piece_id=L4YHP36262XOVY2LYVY7XLEO426GL4MUQOSBUWMPER5OJSZELEIA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.457086718 (13h9m56.682269775s) [noise=true , address=45.91.116.10:4096 , piece_id=5UFXZTBHAHGO66MIKNHEYQDHR6TU3FPSHS4QAEWCWLKB7KS5E3VQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.457243393 (13h9m56.68242644s) [noise=true , address=45.91.116.12:4251 , piece_id=LATWDNZP36TIMTDXPCUZF7BOMCRPIS5IZ4PPXZFAPE4PLRHTY3ZQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.457840053 (13h9m56.68302311s) [noise=true , address=45.91.116.5:4310 , piece_id=JWV5PSICVRAUEHE2ZMPSLZQHF6YW4UFPOOGUX2RVKCY7CFSARXBQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.459597013 (13h9m56.68478007s) [noise=true , address=77.90.31.21:4881 , piece_id=6UQY5WOUIA2J6U75DBATVIMBTKF6VKOG5JHFMICL6YNF6MGFFZJQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.460289213 (13h9m56.685472261s) [noise=true , address=77.90.31.13:4640 , piece_id=YL5GO7EZG6NEUAV6W7VNLUVMC6COHESWJAL7SOKSMHNDDZPGVSXQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.460768433 (13h9m56.685951481s) [noise=true , address=77.90.31.19:4852 , piece_id=I22RF3HMIKEKYKHPRXRYN6KW3OMFDN646KC4HR24DQL23SOJRBOQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.461901491 (13h9m56.687084538s) [noise=true , address=104.36.82.13:4203 , piece_id=EZS5HYDFGSVI6LMISBGO3N4QCAM7GWZUCTLIL6JXIJAIEPLW62FA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.461909817 (13h9m56.687092874s) [noise=true , address=104.36.82.5:4869 , piece_id=QCCC7642LGPS5RXY7DAHDGPZTNGLASMHXZQP7KSVLYCOOXCXNLFQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.462644336 (13h9m56.687827383s) [noise=true , address=104.36.82.44:4899 , piece_id=5GUG5G27NPKBMMP6UPANTMOVHGWIZOQCFQIMNUAXZNLADOQOHKNQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.469009209 (13h9m56.694192267s) [noise=true , address=104.36.82.15:4292 , piece_id=A7A56VDLWRDCIU72YQUUPUFZ7AHSYTZQK7SSUQOG6ZPSLQARI7VA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.485238089 (13h9m56.710421136s) [noise=true , address=104.36.82.60:4368 , piece_id=QDAYLFDR53XS5JW7UQRTA7QVCRIUDD2DVQXJEFPHH6PXTXFHZ7XQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.606276829 (13h9m56.831459876s) [noise=true , address=79.127.201.214:4299 , piece_id=XA5YQKO3E6P537ANACWNTISC3KM7XTNP7DWL2JYHHUIYQOX4UWPA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.622092552 (13h9m56.84727563s) [noise=true , address=79.127.201.215:4343 , piece_id=NGLZCPZSWNVRU2EOKURK7EYIBBQ5USTN4EIB34AXJWPWAHDRGBEQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: <nil> Aug 26 05:13:41.622130043 (13h9m56.84731313s) [seg_pos={0 0}, upload=11565]: Segment reached optimal threshold of 14 pieces. Aug 26 05:13:41.622992032 (13h9m56.84817511s) [noise=true , address=64.49.13.140:4508 , piece_id=G7Y7QOFOOMRSVVELUCHKK4IGX6NPPKPZPKHRN2G7DSWLYVETSKJQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1diUU2skKzXPjYy3Mu3iDDjnG5W8Z2cxCRMxDCH9cQGg2TdEiR): protocol: read tcp 5.161.89.252:56940->64.49.13.140:4508: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623005398 (13h9m56.848188474s) [noise=true , address=79.127.205.234:4620 , piece_id=QLOKDNIIFRRTWPRBZBSSYY5LBZ5OE2JAORZBK5R464CMFDUNIZQQ, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CvVNHcgV57c7VTpV97A8kXNuV3grxknxgNL9kJFZo9aKqo6L4): protocol: read tcp 5.161.89.252:38838->79.127.205.234:4620: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623062946 (13h9m56.848246022s) [noise=true , address=193.149.164.143:4694 , piece_id=JQ3CZ3LRSCMD3LTT7P3PVMHDMWMNRBYMZDEDYXXBEF2WZEVIT6QA, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ATtVD3XyFxzEqwQpSF4t1MWygqEa4Y1CkNj1s43eud8UtuSmx): protocol: read tcp 5.161.89.252:46402->193.149.164.143:4694: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623102279 (13h9m56.848285357s) [noise=true , address=79.127.205.246:4254 , piece_id=EW3ACHT6SG4VHW6DSZ2TBI55Q75G6VRNH4X7BFQQCEV6WMVYYW5Q, seg_pos={0 0}, upload=11565]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zrPqM7xhRpeXnvSRp2dssRupk6zgCK7KTJJFxoVQvXGmF5445): protocol: read tcp 5.161.89.252:53940->79.127.205.246:4254: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:13:41.623133287 (13h9m56.848316364s) [seg_pos={0 0}, upload=11565]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:13:41.623206976 (13h9m56.848390062s) [upload=11565]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:13:41.623215051 (13h9m56.848398138s) [upload=11565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:13:41.947765180 (13h9m57.172948258s) [upload=11565]: Done uploading segments... Aug 26 05:13:41.947795217 (13h9m57.172978294s) [upload=11565]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 697.416673ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................X..................XXX....X] duration: 6.400759768s FAIL (2025-08-26 05:13:31 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F392527B5AFE0, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.400783904s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X.X..] duration: 39.438130279s FAIL (2025-08-26 05:13:23 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3905DDF95AE1, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 39.438148042s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [............................................X..........X....] duration: 3.854973873s remote: "tcp:136.0.77.2:443" response headers: Server=Storj X-Amz-Request-Id=185F38ED4FE1C41F X-Request-Id=AxxP2Ltro5h Date=Tue, 26 Aug 2025 05:09:31 GMT Content-Length=423 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Type=application/xml target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:13:21 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F38ED4FE1C41F</RequestId><HostId></HostId></Error>" Duration: 3.854996775s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..................................X.....X...........X.X.XX.X] duration: 8.430915278s FAIL (2025-08-26 05:13:19 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:13:11.244709301 (13h9m26.469892368s) [upload=11557]: Starting upload Aug 26 05:13:11.244824367 (13h9m26.470007444s) [upload=11557]: Done starting upload Aug 26 05:13:11.244917071 (13h9m26.470100149s) [upload=11557]: Uploading segments... Aug 26 05:13:11.245212917 (13h9m26.470395994s) [upload=11557]: Got next segment. Inline: false Aug 26 05:13:11.245257991 (13h9m26.470441058s) [upload=11557]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:13:11.245263852 (13h9m26.47044693s) [upload=11557]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:13:14.827341263 (13h9m30.05252434s) [seg_pos={0 0}, upload=11557]: Begin upload segment... Aug 26 05:13:14.827380065 (13h9m30.052563143s) [seg_pos={0 0}, upload=11557]: Done begin upload segment. Aug 26 05:13:14.827406485 (13h9m30.052589562s) [noise=true , address=77.90.31.11:4555 , piece_id=OSCIFNYZQHRWHWBXUNETRLENGSN2ADCN4QMHHCJ26AY45CTOZAFA, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.827450307 (13h9m30.052633385s) [noise=true , address=104.36.82.2:4732 , piece_id=VT3HE446IJJ2BBJEXJWFZDARW7UQ4T5VFKQ7SV4J2E6OFYGXS6EQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.827853815 (13h9m30.053036892s) [noise=true , address=45.91.116.10:4081 , piece_id=ZO46LOCYLTKVM4ATRYEO7GHJADL6ZDKUATI4JAOGISNRH4AX27JQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.828037671 (13h9m30.053220738s) [noise=true , address=77.90.31.11:4554 , piece_id=7RGQCXBH25YIKD4HQGZ66EKVEMMNXZVDEKDORAYWZMAAXXPYVV6A, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.828265648 (13h9m30.053448726s) [noise=true , address=77.90.31.15:4698 , piece_id=MI3UYSBH4WM5SMMI4HNDW2DSGQJDFJVRZBPFTFW6S7L6M5MOLWYQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.828408457 (13h9m30.053591534s) [noise=true , address=45.91.116.9:4263 , piece_id=KVU4CF7VCDSQ5T4K75COYSSXLM3HEBIQR3MB6OXHLAYZEJ66LQ4A, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.828621997 (13h9m30.053805075s) [noise=true , address=77.90.31.19:4842 , piece_id=ANA57Z7RRHA52KORIZZVSJWPCNORHZWQG4TAKCMNS4YVLNRVP2SQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.828816513 (13h9m30.053999591s) [noise=true , address=45.91.116.14:4190 , piece_id=OGDKGMGBLF4NLF2QMJMLOVQTOXFHIB35N7L7HV7TSBG54WOHWPTQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.829013132 (13h9m30.05419621s) [noise=true , address=45.91.116.10:4083 , piece_id=ZYDI5WZ3JF4GD2MCWJVTDCIGYMZ7MWSY6UCZALFL6JWGY2U3GTPA, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.829236883 (13h9m30.05441996s) [noise=true , address=45.91.116.7:4009 , piece_id=6K2X6UP7SHJLGGRHTS2OMOTUNJKUHXUHY3LZUN4BOIESNE2QEDXA, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.829374061 (13h9m30.054557137s) [noise=true , address=45.91.116.10:4082 , piece_id=YMDLM7P6PCMHX4ECNHGEMSBVKBPSMLFRDFOI2PAKVALYOOB3PEDQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.829639499 (13h9m30.054822577s) [noise=true , address=45.91.116.14:4191 , piece_id=HDTBI7CS37GRS4TBHJYDBEI6HUTGDBFT6WFKCCPRNBJQLUG5WA4Q, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.829702999 (13h9m30.054886075s) [noise=true , address=77.90.31.14:4663 , piece_id=M7LR4NTYZJ2GV6TM2VMN24B26D3BDAGOTDBARG6MSDVIFS3M55AQ, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.830050721 (13h9m30.055233808s) [noise=true , address=77.90.31.21:4877 , piece_id=2B4J4IYIZ3ITBKOKAZ6RBGUGDIWSN7THZE3IWP7NBEZBYIHGXH7Q, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.830116014 (13h9m30.055299081s) [noise=true , address=45.91.116.12:4225 , piece_id=UUY5DN54ZVJUQGHKFAV373K3QOAKPMZXFQZYQSQ6AUF4YC3S22XA, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.830470910 (13h9m30.055653978s) [noise=true , address=104.36.82.12:4150 , piece_id=XF3PWINBYCR77F7WARG2W7VDDXEI73VE2YZMXDFB3TKWIG56DXDA, seg_pos={0 0}, upload=11557]: Uploading piece... Aug 26 05:13:14.937354316 (13h9m30.162537373s) [upload=11557]: Next returned nil segment Aug 26 05:13:14.937365857 (13h9m30.162548914s) [upload=11557]: Waiting for error group managing segments... Aug 26 05:13:14.946997527 (13h9m30.172180584s) [noise=true , address=45.91.116.12:4225 , piece_id=UUY5DN54ZVJUQGHKFAV373K3QOAKPMZXFQZYQSQ6AUF4YC3S22XA, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.947208132 (13h9m30.172391189s) [noise=true , address=45.91.116.10:4083 , piece_id=ZYDI5WZ3JF4GD2MCWJVTDCIGYMZ7MWSY6UCZALFL6JWGY2U3GTPA, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.947718059 (13h9m30.172901106s) [noise=true , address=45.91.116.14:4190 , piece_id=OGDKGMGBLF4NLF2QMJMLOVQTOXFHIB35N7L7HV7TSBG54WOHWPTQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.947993076 (13h9m30.173176133s) [noise=true , address=45.91.116.14:4191 , piece_id=HDTBI7CS37GRS4TBHJYDBEI6HUTGDBFT6WFKCCPRNBJQLUG5WA4Q, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.948196087 (13h9m30.173379144s) [noise=true , address=45.91.116.10:4082 , piece_id=YMDLM7P6PCMHX4ECNHGEMSBVKBPSMLFRDFOI2PAKVALYOOB3PEDQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.948806704 (13h9m30.173989761s) [noise=true , address=45.91.116.10:4081 , piece_id=ZO46LOCYLTKVM4ATRYEO7GHJADL6ZDKUATI4JAOGISNRH4AX27JQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.949073906 (13h9m30.174256963s) [noise=true , address=77.90.31.11:4554 , piece_id=7RGQCXBH25YIKD4HQGZ66EKVEMMNXZVDEKDORAYWZMAAXXPYVV6A, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.949210823 (13h9m30.17439388s) [noise=true , address=77.90.31.14:4663 , piece_id=M7LR4NTYZJ2GV6TM2VMN24B26D3BDAGOTDBARG6MSDVIFS3M55AQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.949760486 (13h9m30.174943543s) [noise=true , address=77.90.31.21:4877 , piece_id=2B4J4IYIZ3ITBKOKAZ6RBGUGDIWSN7THZE3IWP7NBEZBYIHGXH7Q, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.950227242 (13h9m30.175410299s) [noise=true , address=45.91.116.9:4263 , piece_id=KVU4CF7VCDSQ5T4K75COYSSXLM3HEBIQR3MB6OXHLAYZEJ66LQ4A, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.950598219 (13h9m30.175781276s) [noise=true , address=45.91.116.7:4009 , piece_id=6K2X6UP7SHJLGGRHTS2OMOTUNJKUHXUHY3LZUN4BOIESNE2QEDXA, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.951017246 (13h9m30.176200303s) [noise=true , address=77.90.31.19:4842 , piece_id=ANA57Z7RRHA52KORIZZVSJWPCNORHZWQG4TAKCMNS4YVLNRVP2SQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.951369758 (13h9m30.176552805s) [noise=true , address=77.90.31.15:4698 , piece_id=MI3UYSBH4WM5SMMI4HNDW2DSGQJDFJVRZBPFTFW6S7L6M5MOLWYQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: <nil> Aug 26 05:13:14.951375899 (13h9m30.176558956s) [seg_pos={0 0}, upload=11557]: Segment reached optimal threshold of 13 pieces. Aug 26 05:13:14.951504380 (13h9m30.176687437s) [noise=true , address=104.36.82.2:4732 , piece_id=VT3HE446IJJ2BBJEXJWFZDARW7UQ4T5VFKQ7SV4J2E6OFYGXS6EQ, seg_pos={0 0}, upload=11557]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 26 05:13:14.951593519 (13h9m30.176776576s) [noise=true , address=104.36.82.12:4150 , piece_id=XF3PWINBYCR77F7WARG2W7VDDXEI73VE2YZMXDFB3TKWIG56DXDA, seg_pos={0 0}, upload=11557]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Aug 26 05:13:14.951692705 (13h9m30.176875751s) [noise=true , address=77.90.31.11:4555 , piece_id=OSCIFNYZQHRWHWBXUNETRLENGSN2ADCN4QMHHCJ26AY45CTOZAFA, seg_pos={0 0}, upload=11557]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: expected piece hash; context canceled; EOF Aug 26 05:13:14.951703665 (13h9m30.176886722s) [seg_pos={0 0}, upload=11557]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:13:14.951755242 (13h9m30.176938299s) [upload=11557]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:13:14.951758638 (13h9m30.176941686s) [upload=11557]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:13:19.658945594 (13h9m34.884128651s) [upload=11557]: Done uploading segments... Aug 26 05:13:19.658971824 (13h9m34.884154881s) [upload=11557]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 8.430932009s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.................................X.....XX..........XX....X..] duration: 7.119118953s FAIL (2025-08-26 05:13:19 UTC) operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 7.119138259s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................XX.......X..X.......XXXXXXX] duration: 30.00750097s FAIL (2025-08-26 05:13:07 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.007526981s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X..................XXX....] duration: 9.33370236s FAIL (2025-08-26 05:13:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.333749099s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)