Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.........X............................................] duration: 11.143595752s FAIL (2025-08-16 16:11:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 11.143619156s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........XX.X......X......................................] duration: 441.073074ms FAIL (2025-08-16 16:11:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 441.090617ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 122.396315ms FAIL (2025-08-16 16:10:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f36581b1eb373626007067365b048f5ce5cc5372ae2c12cedc5ebb9fac1fe3b1 != before 5103a32e94533b210bafbc980191e5d65541c62ead63349960a29c0fe846ecfc ======LOGS====== Aug 16 16:10:27.681038132 (190h44m8.412305783s) [upload=198465]: Starting upload Aug 16 16:10:27.681106641 (190h44m8.412374291s) [upload=198465]: Done starting upload Aug 16 16:10:27.681180781 (190h44m8.412448431s) [upload=198465]: Uploading segments... Aug 16 16:10:27.681239091 (190h44m8.41250674s) [upload=198465]: Got next segment. Inline: true Aug 16 16:10:27.681246394 (190h44m8.412514045s) [upload=198465]: Waiting for error group managing segments... Aug 16 16:10:27.681294365 (190h44m8.412562016s) [upload=198465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:10:27.681300507 (190h44m8.412568157s) [upload=198465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:10:27.681304214 (190h44m8.412571864s) [upload=198465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:10:44.137637975 (190h44m24.868905625s) [upload=198465]: Done uploading segments... Aug 16 16:10:44.137657582 (190h44m24.868925242s) [upload=198465]: Upload finished. err: <nil> Duration:* 122.414009ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.........X...........................................] duration: 13.021341056s FAIL (2025-08-16 16:10:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 13.021366123s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 16.868578685s FAIL (2025-08-16 16:10:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f36581b1eb373626007067365b048f5ce5cc5372ae2c12cedc5ebb9fac1fe3b1 != before 5103a32e94533b210bafbc980191e5d65541c62ead63349960a29c0fe846ecfc ======LOGS====== Aug 16 16:10:27.681038132 (190h44m8.412305783s) [upload=198465]: Starting upload Aug 16 16:10:27.681106641 (190h44m8.412374291s) [upload=198465]: Done starting upload Aug 16 16:10:27.681180781 (190h44m8.412448431s) [upload=198465]: Uploading segments... Aug 16 16:10:27.681239091 (190h44m8.41250674s) [upload=198465]: Got next segment. Inline: true Aug 16 16:10:27.681246394 (190h44m8.412514045s) [upload=198465]: Waiting for error group managing segments... Aug 16 16:10:27.681294365 (190h44m8.412562016s) [upload=198465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 16:10:27.681300507 (190h44m8.412568157s) [upload=198465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 16:10:27.681304214 (190h44m8.412571864s) [upload=198465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 16:10:44.137637975 (190h44m24.868905625s) [upload=198465]: Done uploading segments... Aug 16 16:10:44.137657582 (190h44m24.868925242s) [upload=198465]: Upload finished. err: <nil> Duration:* 16.868604483s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............XX.X......X.....................................] duration: 409.627787ms FAIL (2025-08-16 16:10:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 409.64592ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.........X..........................................] duration: 507.533164ms FAIL (2025-08-16 16:09:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 507.550266ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............XX.X......X....................................] duration: 420.29912ms FAIL (2025-08-16 16:09:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 420.323495ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X.........X.........................................] duration: 10.34474511s FAIL (2025-08-16 16:08:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.344765208s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............XX.X......X...................................] duration: 383.642804ms FAIL (2025-08-16 16:08:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 383.671158ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X.........X........................................] duration: 18.968136871s FAIL (2025-08-16 16:07:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.968160276s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............XX.X......X..................................] duration: 529.155562ms FAIL (2025-08-16 16:07:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 529.181571ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.........X.........X.......................................] duration: 27.056843767s FAIL (2025-08-16 16:06:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.056872603s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................XX.X......X.................................] duration: 401.101558ms FAIL (2025-08-16 16:06:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 401.119932ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................XX.X......X................................] duration: 561.427945ms FAIL (2025-08-16 16:06:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 561.448594ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [XX.........X.........X......................................] duration: 21.320774529s FAIL (2025-08-16 16:05:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 21.320914253s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XX.........X.........X.....................................] duration: 27.545248615s FAIL (2025-08-16 16:05:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.545267511s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................XX.X......X...............................] duration: 388.642275ms FAIL (2025-08-16 16:04:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 388.66657ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 352.735428ms FAIL (2025-08-16 16:04:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 352.807114ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 32.165915241s FAIL (2025-08-16 16:04:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 32.165933616s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XX.........X.........X....................................] duration: 14.488400358s FAIL (2025-08-16 16:03:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.488424534s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................XX.X......X..............................] duration: 674.000515ms FAIL (2025-08-16 16:03:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 674.018208ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XX.........X.........X...................................] duration: 19.93242047s FAIL (2025-08-16 16:03:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.932445437s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 7.286347189s FAIL (2025-08-16 16:03:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 7.286436978s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XX.........X.........X..................................] duration: 933.282124ms FAIL (2025-08-16 16:02:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 933.301161ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XX.X......X.............................] duration: 1.009367452s FAIL (2025-08-16 16:02:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.009415474s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 11.565108971s FAIL (2025-08-16 16:02:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 11.565130963s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....XX.........X.........X.................................] duration: 25.241031714s FAIL (2025-08-16 16:01:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.241050519s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XX.X......X............................] duration: 486.661493ms FAIL (2025-08-16 16:01:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 486.695528ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XX.X......X...........................] duration: 551.246446ms FAIL (2025-08-16 16:01:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 551.265922ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XX.X......X..........................] duration: 6.377653124s FAIL (2025-08-16 16:00:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 6.377675475s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 1.780774589s FAIL (2025-08-16 16:00:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 1.780795519s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......XX.........X.........X................................] duration: 13.996587857s FAIL (2025-08-16 16:00:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 13.996612732s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XX.X......X.........................] duration: 8.708532513s FAIL (2025-08-16 16:00:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 8.70856297s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......XX.........X.........X...............................] duration: 34.498037318s FAIL (2025-08-16 15:59:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 34.498060281s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...X......................................................] duration: 507.391311ms FAIL (2025-08-16 15:59:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 507.412512ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XX.X......X........................] duration: 1.118540653s FAIL (2025-08-16 15:59:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.118593293s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........XX.........X.........X..............................] duration: 27.517480527s FAIL (2025-08-16 15:58:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.51749228s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...X.....................................................] duration: 528.656189ms FAIL (2025-08-16 15:58:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 528.680805ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........XX.........X.........X.............................] duration: 37.819844296s FAIL (2025-08-16 15:58:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 37.819889582s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XX.X......X.......................] duration: 864.673484ms FAIL (2025-08-16 15:58:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 864.69753ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XX.X......X......................] duration: 664.868581ms FAIL (2025-08-16 15:57:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 664.886163ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........XX.........X.........X............................] duration: 22.626293925s FAIL (2025-08-16 15:57:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 22.626308843s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...X....................................................] duration: 314.588537ms FAIL (2025-08-16 15:57:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 314.607613ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X...X...................................................] duration: 836.371619ms FAIL (2025-08-16 15:57:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 836.396915ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........XX.........X.........X...........................] duration: 25.893336264s FAIL (2025-08-16 15:56:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.893353977s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XX.X......X.....................] duration: 425.908749ms FAIL (2025-08-16 15:56:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 425.938385ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............XX.........X.........X..........................] duration: 17.791126756s FAIL (2025-08-16 15:55:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.791147073s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X...X..................................................] duration: 851.752148ms FAIL (2025-08-16 15:55:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 851.765443ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XX.X......X....................] duration: 1.675734578s FAIL (2025-08-16 15:55:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.67576242s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)