Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X...X................................] duration: 290.712932ms FAIL (2025-08-16 15:42:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 290.735174ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X.................XX.........X.........X..........] duration: 17.593299631s FAIL (2025-08-16 15:42:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.593320781s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XX.X......X.] duration: 25.953812807s FAIL (2025-08-16 15:42:16 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 25.953833235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X...X...............................] duration: 512.138467ms FAIL (2025-08-16 15:42:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 512.165978ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X...X..............................] duration: 155.627018ms FAIL (2025-08-16 15:41:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 155.645374ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X...X.............................] duration: 920.745907ms FAIL (2025-08-16 15:41:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 920.771815ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX.X......X] duration: 30.001768763s FAIL (2025-08-16 15:41:42 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 30.001786646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X.................XX.........X.........X.........] duration: 21.670625322s FAIL (2025-08-16 15:41:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 21.670656278s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X...X............................] duration: 757.548768ms FAIL (2025-08-16 15:41:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 757.569185ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X...X...........................] duration: 1.372187746s FAIL (2025-08-16 15:40:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 1.372210568s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X.................XX.........X.........X........] duration: 10.511017856s FAIL (2025-08-16 15:40:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.511057701s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XX.X......] duration: 8.468589358s FAIL (2025-08-16 15:40:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.468615668s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X...X..........................] duration: 1.978481494s FAIL (2025-08-16 15:40:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 1.97851662s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XX.X.....] duration: 10.826029804s FAIL (2025-08-16 15:39:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 10.826048609s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X...X.........................] duration: 4.28783436s FAIL (2025-08-16 15:39:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 4.28785575s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X.................XX.........X.........X.......] duration: 15.474013178s FAIL (2025-08-16 15:39:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.474030401s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X.................XX.........X.........X......] duration: 37.498385482s FAIL (2025-08-16 15:39:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 37.498400671s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XX.X....] duration: 20.095091228s FAIL (2025-08-16 15:39:02 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 20.095115203s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..............X.................XX.........X.........X.....] duration: 19.20329193s FAIL (2025-08-16 15:38:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.203323099s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X...X........................] duration: 1.149372096s FAIL (2025-08-16 15:38:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 1.149392463s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XX.X...] duration: 13.364953846s FAIL (2025-08-16 15:38:28 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 13.364981177s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X..............X.................XX.........X.........X....] duration: 21.292568961s FAIL (2025-08-16 15:38:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 21.292586914s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XX.X..] duration: 3.593278119s FAIL (2025-08-16 15:37:55 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.5932998s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X...X.......................] duration: 259.57892ms FAIL (2025-08-16 15:37:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 259.603986ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XX.X.] duration: 460.995389ms FAIL (2025-08-16 15:37:29 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 461.018702ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XX.X] duration: 30.015775487s FAIL (2025-08-16 15:37:25 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.015794302s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X..............X.................XX.........X.........X...] duration: 24.383840525s FAIL (2025-08-16 15:37:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 24.3838595s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X...X......................] duration: 164.905442ms FAIL (2025-08-16 15:36:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 164.949155ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X..............X.................XX.........X.........X..] duration: 27.444512184s FAIL (2025-08-16 15:36:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.444588789s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 8.923488815s FAIL (2025-08-16 15:36:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:35:40.865505347 (190h9m21.596772977s) [upload=197865]: Starting upload Aug 16 15:35:40.865567603 (190h9m21.596835233s) [upload=197865]: Done starting upload Aug 16 15:35:40.865643696 (190h9m21.596911326s) [upload=197865]: Uploading segments... Aug 16 15:35:40.865823915 (190h9m21.597091535s) [upload=197865]: Got next segment. Inline: false Aug 16 15:35:40.865881563 (190h9m21.597149193s) [upload=197865]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:35:40.865889027 (190h9m21.597156657s) [upload=197865]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:35:40.889812243 (190h9m21.621079863s) [seg_pos={0 0}, upload=197865]: Begin upload segment... Aug 16 15:35:40.889845325 (190h9m21.621112945s) [seg_pos={0 0}, upload=197865]: Done begin upload segment. Aug 16 15:35:40.889865333 (190h9m21.621132963s) [noise=true , address=77.90.31.13:4628 , piece_id=2WJEQGMEZHOCIJMLQQSWS2M5TTSEWTSEO7SZ45HWSND4UEUBWL4Q, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.889882815 (190h9m21.621150435s) [noise=true , address=104.36.82.7:4912 , piece_id=MDKARZVPP6F5RQILZLGCHNNQNWAM3EIMYBPXUBQPB5PTK4ULWJKA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890219348 (190h9m21.621486967s) [noise=true , address=104.36.82.14:4223 , piece_id=YEXBFKLHY7JUSFPKE3HXPUNDQJW2M7BP5ZGMBKDAMA6YP4LFWCUQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890227243 (190h9m21.621494862s) [noise=true , address=45.91.116.13:4155 , piece_id=UT25PDUTGDI7VG2JJNL233IP4JPYQQRRTKPBDOYBVQO23YJIQGUQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890476981 (190h9m21.621744602s) [noise=true , address=77.90.31.11:4555 , piece_id=5M3E237EQJ7U2FQD4TUQEJWLDWZCQZZSWAKFZQQEEZDG4AQGQRHA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890485227 (190h9m21.621752847s) [noise=true , address=45.91.116.4:4940 , piece_id=IGC74AMGEWP54OKJRKFMPU6HIYC4DKSWPOFLDLDPD5AX443NQFNA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890720630 (190h9m21.62198826s) [noise=true , address=77.90.31.12:4589 , piece_id=XNLY7ZWI4FQMYQ35YQL7JDHVEQHES4M5HEBPEWOCZVXYZONSUUGQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890956614 (190h9m21.622224244s) [noise=true , address=104.36.82.3:4767 , piece_id=QZDFS2NVPXOYUMRP425IWAFDSFZMC74EEB4FR4T5XUDKSWO2KMXQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.891300620 (190h9m21.62256825s) [noise=true , address=45.91.116.10:4081 , piece_id=N5CIGBUUFBYBVYIEWX7HYKNKBVURXLPDUCWIWW7R66BW6SAPFIZA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.891578171 (190h9m21.622845791s) [noise=true , address=77.90.31.18:4807 , piece_id=VJQHWT3Y33YEBLCC7TARWJ45LX462MBD2UJXNAF7M6PN7ZSU2S3A, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.891824404 (190h9m21.623092024s) [noise=true , address=104.36.82.16:4294 , piece_id=U6TVGYQCZ32I3ZJ4VSC7F4WM2NDNG6N4H54Z6XJR2WPTFBDFEQIQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892064024 (190h9m21.623331654s) [noise=true , address=45.91.116.5:4297 , piece_id=22NYVN53L4LKL2ZRUR7EQLYUNISDV5MAQ4NXLSA52NFJPKGFS6CA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892492059 (190h9m21.623759689s) [noise=true , address=104.36.82.5:4840 , piece_id=UNUUPYO6XREUE3MM5Z3QDA5ZFL4R6PLW6ZJK4HEFJXIZXLNGQRHA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892716681 (190h9m21.623984311s) [noise=true , address=77.90.31.13:4626 , piece_id=TKRKB4R54IABM6FSAACU3645QYDGLQPIOJ5C4CXHCMY7PK7ZI5EQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892924030 (190h9m21.62419165s) [noise=true , address=45.91.116.12:4227 , piece_id=26HMSZUXLM6NNESYMVG3ZTTE2INJFJQPVHKE2POJ272Z3WZEI65Q, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.893367945 (190h9m21.624635564s) [noise=true , address=45.91.116.12:4226 , piece_id=TJNCDKHTYM2QPRTAPG7X6ARLGXKBWVHITTQ3DVANKL5S326MBQMA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.969805832 (190h9m21.701073453s) [upload=197865]: Next returned nil segment Aug 16 15:35:40.969817775 (190h9m21.701085405s) [upload=197865]: Waiting for error group managing segments... Aug 16 15:35:40.987547271 (190h9m21.718814891s) [noise=true , address=104.36.82.7:4912 , piece_id=MDKARZVPP6F5RQILZLGCHNNQNWAM3EIMYBPXUBQPB5PTK4ULWJKA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.255536594 (190h9m21.986804225s) [noise=true , address=45.91.116.13:4155 , piece_id=UT25PDUTGDI7VG2JJNL233IP4JPYQQRRTKPBDOYBVQO23YJIQGUQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.258540489 (190h9m21.989808109s) [noise=true , address=77.90.31.18:4807 , piece_id=VJQHWT3Y33YEBLCC7TARWJ45LX462MBD2UJXNAF7M6PN7ZSU2S3A, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.260000753 (190h9m21.991268373s) [noise=true , address=77.90.31.12:4589 , piece_id=XNLY7ZWI4FQMYQ35YQL7JDHVEQHES4M5HEBPEWOCZVXYZONSUUGQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.410198948 (190h9m22.141466568s) [noise=true , address=45.91.116.4:4940 , piece_id=IGC74AMGEWP54OKJRKFMPU6HIYC4DKSWPOFLDLDPD5AX443NQFNA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.613938144 (190h9m22.345205774s) [noise=true , address=45.91.116.10:4081 , piece_id=N5CIGBUUFBYBVYIEWX7HYKNKBVURXLPDUCWIWW7R66BW6SAPFIZA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:42.076438272 (190h9m22.807705902s) [noise=true , address=45.91.116.12:4226 , piece_id=TJNCDKHTYM2QPRTAPG7X6ARLGXKBWVHITTQ3DVANKL5S326MBQMA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:43.085963451 (190h9m23.817231101s) [noise=true , address=45.91.116.12:4227 , piece_id=26HMSZUXLM6NNESYMVG3ZTTE2INJFJQPVHKE2POJ272Z3WZEI65Q, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:43.300280055 (190h9m24.031547685s) [noise=true , address=104.36.82.5:4840 , piece_id=UNUUPYO6XREUE3MM5Z3QDA5ZFL4R6PLW6ZJK4HEFJXIZXLNGQRHA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:44.181975802 (190h9m24.913243423s) [noise=true , address=45.91.116.5:4297 , piece_id=22NYVN53L4LKL2ZRUR7EQLYUNISDV5MAQ4NXLSA52NFJPKGFS6CA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:56.370155926 (190h9m37.101423556s) [noise=true , address=77.90.31.11:4555 , piece_id=5M3E237EQJ7U2FQD4TUQEJWLDWZCQZZSWAKFZQQEEZDG4AQGQRHA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:36:05.254311274 (190h9m45.985578904s) [noise=true , address=104.36.82.14:4223 , piece_id=YEXBFKLHY7JUSFPKE3HXPUNDQJW2M7BP5ZGMBKDAMA6YP4LFWCUQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:36:10.853973711 (190h9m51.585241361s) [noise=true , address=104.36.82.3:4767 , piece_id=QZDFS2NVPXOYUMRP425IWAFDSFZMC74EEB4FR4T5XUDKSWO2KMXQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:36:10.854130246 (190h9m51.585397906s) [noise=true , address=77.90.31.13:4626 , piece_id=TKRKB4R54IABM6FSAACU3645QYDGLQPIOJ5C4CXHCMY7PK7ZI5EQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:25796->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:36:10.854199095 (190h9m51.585466744s) [noise=true , address=104.36.82.16:4294 , piece_id=U6TVGYQCZ32I3ZJ4VSC7F4WM2NDNG6N4H54Z6XJR2WPTFBDFEQIQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:54190->104.36.82.16:4294: use of closed network connection; write tcp 5.161.89.252:54190->104.36.82.16:4294: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54190->104.36.82.16:4294: use of closed network connection Aug 16 15:36:10.854299855 (190h9m51.585567505s) [noise=true , address=77.90.31.13:4628 , piece_id=2WJEQGMEZHOCIJMLQQSWS2M5TTSEWTSEO7SZ45HWSND4UEUBWL4Q, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:36:10.854333859 (190h9m51.585601518s) [seg_pos={0 0}, upload=197865]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:36:10.871068726 (190h9m51.602336386s) [upload=197865]: Done uploading segments... Aug 16 15:36:10.871094925 (190h9m51.602362575s) [upload=197865]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.92350198s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X...X.....................] duration: 722.27833ms FAIL (2025-08-16 15:36:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 722.303037ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.018959643s FAIL (2025-08-16 15:36:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:35:40.865505347 (190h9m21.596772977s) [upload=197865]: Starting upload Aug 16 15:35:40.865567603 (190h9m21.596835233s) [upload=197865]: Done starting upload Aug 16 15:35:40.865643696 (190h9m21.596911326s) [upload=197865]: Uploading segments... Aug 16 15:35:40.865823915 (190h9m21.597091535s) [upload=197865]: Got next segment. Inline: false Aug 16 15:35:40.865881563 (190h9m21.597149193s) [upload=197865]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:35:40.865889027 (190h9m21.597156657s) [upload=197865]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:35:40.889812243 (190h9m21.621079863s) [seg_pos={0 0}, upload=197865]: Begin upload segment... Aug 16 15:35:40.889845325 (190h9m21.621112945s) [seg_pos={0 0}, upload=197865]: Done begin upload segment. Aug 16 15:35:40.889865333 (190h9m21.621132963s) [noise=true , address=77.90.31.13:4628 , piece_id=2WJEQGMEZHOCIJMLQQSWS2M5TTSEWTSEO7SZ45HWSND4UEUBWL4Q, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.889882815 (190h9m21.621150435s) [noise=true , address=104.36.82.7:4912 , piece_id=MDKARZVPP6F5RQILZLGCHNNQNWAM3EIMYBPXUBQPB5PTK4ULWJKA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890219348 (190h9m21.621486967s) [noise=true , address=104.36.82.14:4223 , piece_id=YEXBFKLHY7JUSFPKE3HXPUNDQJW2M7BP5ZGMBKDAMA6YP4LFWCUQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890227243 (190h9m21.621494862s) [noise=true , address=45.91.116.13:4155 , piece_id=UT25PDUTGDI7VG2JJNL233IP4JPYQQRRTKPBDOYBVQO23YJIQGUQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890476981 (190h9m21.621744602s) [noise=true , address=77.90.31.11:4555 , piece_id=5M3E237EQJ7U2FQD4TUQEJWLDWZCQZZSWAKFZQQEEZDG4AQGQRHA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890485227 (190h9m21.621752847s) [noise=true , address=45.91.116.4:4940 , piece_id=IGC74AMGEWP54OKJRKFMPU6HIYC4DKSWPOFLDLDPD5AX443NQFNA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890720630 (190h9m21.62198826s) [noise=true , address=77.90.31.12:4589 , piece_id=XNLY7ZWI4FQMYQ35YQL7JDHVEQHES4M5HEBPEWOCZVXYZONSUUGQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.890956614 (190h9m21.622224244s) [noise=true , address=104.36.82.3:4767 , piece_id=QZDFS2NVPXOYUMRP425IWAFDSFZMC74EEB4FR4T5XUDKSWO2KMXQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.891300620 (190h9m21.62256825s) [noise=true , address=45.91.116.10:4081 , piece_id=N5CIGBUUFBYBVYIEWX7HYKNKBVURXLPDUCWIWW7R66BW6SAPFIZA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.891578171 (190h9m21.622845791s) [noise=true , address=77.90.31.18:4807 , piece_id=VJQHWT3Y33YEBLCC7TARWJ45LX462MBD2UJXNAF7M6PN7ZSU2S3A, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.891824404 (190h9m21.623092024s) [noise=true , address=104.36.82.16:4294 , piece_id=U6TVGYQCZ32I3ZJ4VSC7F4WM2NDNG6N4H54Z6XJR2WPTFBDFEQIQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892064024 (190h9m21.623331654s) [noise=true , address=45.91.116.5:4297 , piece_id=22NYVN53L4LKL2ZRUR7EQLYUNISDV5MAQ4NXLSA52NFJPKGFS6CA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892492059 (190h9m21.623759689s) [noise=true , address=104.36.82.5:4840 , piece_id=UNUUPYO6XREUE3MM5Z3QDA5ZFL4R6PLW6ZJK4HEFJXIZXLNGQRHA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892716681 (190h9m21.623984311s) [noise=true , address=77.90.31.13:4626 , piece_id=TKRKB4R54IABM6FSAACU3645QYDGLQPIOJ5C4CXHCMY7PK7ZI5EQ, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.892924030 (190h9m21.62419165s) [noise=true , address=45.91.116.12:4227 , piece_id=26HMSZUXLM6NNESYMVG3ZTTE2INJFJQPVHKE2POJ272Z3WZEI65Q, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.893367945 (190h9m21.624635564s) [noise=true , address=45.91.116.12:4226 , piece_id=TJNCDKHTYM2QPRTAPG7X6ARLGXKBWVHITTQ3DVANKL5S326MBQMA, seg_pos={0 0}, upload=197865]: Uploading piece... Aug 16 15:35:40.969805832 (190h9m21.701073453s) [upload=197865]: Next returned nil segment Aug 16 15:35:40.969817775 (190h9m21.701085405s) [upload=197865]: Waiting for error group managing segments... Aug 16 15:35:40.987547271 (190h9m21.718814891s) [noise=true , address=104.36.82.7:4912 , piece_id=MDKARZVPP6F5RQILZLGCHNNQNWAM3EIMYBPXUBQPB5PTK4ULWJKA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.255536594 (190h9m21.986804225s) [noise=true , address=45.91.116.13:4155 , piece_id=UT25PDUTGDI7VG2JJNL233IP4JPYQQRRTKPBDOYBVQO23YJIQGUQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.258540489 (190h9m21.989808109s) [noise=true , address=77.90.31.18:4807 , piece_id=VJQHWT3Y33YEBLCC7TARWJ45LX462MBD2UJXNAF7M6PN7ZSU2S3A, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.260000753 (190h9m21.991268373s) [noise=true , address=77.90.31.12:4589 , piece_id=XNLY7ZWI4FQMYQ35YQL7JDHVEQHES4M5HEBPEWOCZVXYZONSUUGQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.410198948 (190h9m22.141466568s) [noise=true , address=45.91.116.4:4940 , piece_id=IGC74AMGEWP54OKJRKFMPU6HIYC4DKSWPOFLDLDPD5AX443NQFNA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:41.613938144 (190h9m22.345205774s) [noise=true , address=45.91.116.10:4081 , piece_id=N5CIGBUUFBYBVYIEWX7HYKNKBVURXLPDUCWIWW7R66BW6SAPFIZA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:42.076438272 (190h9m22.807705902s) [noise=true , address=45.91.116.12:4226 , piece_id=TJNCDKHTYM2QPRTAPG7X6ARLGXKBWVHITTQ3DVANKL5S326MBQMA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:43.085963451 (190h9m23.817231101s) [noise=true , address=45.91.116.12:4227 , piece_id=26HMSZUXLM6NNESYMVG3ZTTE2INJFJQPVHKE2POJ272Z3WZEI65Q, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:43.300280055 (190h9m24.031547685s) [noise=true , address=104.36.82.5:4840 , piece_id=UNUUPYO6XREUE3MM5Z3QDA5ZFL4R6PLW6ZJK4HEFJXIZXLNGQRHA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:44.181975802 (190h9m24.913243423s) [noise=true , address=45.91.116.5:4297 , piece_id=22NYVN53L4LKL2ZRUR7EQLYUNISDV5MAQ4NXLSA52NFJPKGFS6CA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:35:56.370155926 (190h9m37.101423556s) [noise=true , address=77.90.31.11:4555 , piece_id=5M3E237EQJ7U2FQD4TUQEJWLDWZCQZZSWAKFZQQEEZDG4AQGQRHA, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:36:05.254311274 (190h9m45.985578904s) [noise=true , address=104.36.82.14:4223 , piece_id=YEXBFKLHY7JUSFPKE3HXPUNDQJW2M7BP5ZGMBKDAMA6YP4LFWCUQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: <nil> Aug 16 15:36:10.853973711 (190h9m51.585241361s) [noise=true , address=104.36.82.3:4767 , piece_id=QZDFS2NVPXOYUMRP425IWAFDSFZMC74EEB4FR4T5XUDKSWO2KMXQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:36:10.854130246 (190h9m51.585397906s) [noise=true , address=77.90.31.13:4626 , piece_id=TKRKB4R54IABM6FSAACU3645QYDGLQPIOJ5C4CXHCMY7PK7ZI5EQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:25796->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:36:10.854199095 (190h9m51.585466744s) [noise=true , address=104.36.82.16:4294 , piece_id=U6TVGYQCZ32I3ZJ4VSC7F4WM2NDNG6N4H54Z6XJR2WPTFBDFEQIQ, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:54190->104.36.82.16:4294: use of closed network connection; write tcp 5.161.89.252:54190->104.36.82.16:4294: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54190->104.36.82.16:4294: use of closed network connection Aug 16 15:36:10.854299855 (190h9m51.585567505s) [noise=true , address=77.90.31.13:4628 , piece_id=2WJEQGMEZHOCIJMLQQSWS2M5TTSEWTSEO7SZ45HWSND4UEUBWL4Q, seg_pos={0 0}, upload=197865]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:36:10.854333859 (190h9m51.585601518s) [seg_pos={0 0}, upload=197865]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:36:10.871068726 (190h9m51.602336386s) [upload=197865]: Done uploading segments... Aug 16 15:36:10.871094925 (190h9m51.602362575s) [upload=197865]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.018978498s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X..............X.................XX.........X.........X.] duration: 14.741131528s FAIL (2025-08-16 15:35:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.741149052s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X..............X.................XX.........X.........X] duration: 1m1.682371759s FAIL (2025-08-16 15:35:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m1.682396697s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X...X....................] duration: 6.156349185s FAIL (2025-08-16 15:35:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 6.156371056s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.017099876s FAIL (2025-08-16 15:35:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 16 15:34:51.298323320 (190h8m32.029590971s) [upload=197851]: Starting upload Aug 16 15:34:51.298426874 (190h8m32.029694525s) [upload=197851]: Done starting upload Aug 16 15:34:51.298518267 (190h8m32.029785977s) [upload=197851]: Uploading segments... Aug 16 15:34:51.298803794 (190h8m32.030071453s) [upload=197851]: Got next segment. Inline: false Aug 16 15:34:51.298841404 (190h8m32.030109054s) [upload=197851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:51.298846474 (190h8m32.030114124s) [upload=197851]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:51.323328529 (190h8m32.054596179s) [seg_pos={0 0}, upload=197851]: Begin upload segment... Aug 16 15:34:51.323415391 (190h8m32.054683042s) [seg_pos={0 0}, upload=197851]: Done begin upload segment. Aug 16 15:34:51.323442562 (190h8m32.054710213s) [noise=true , address=104.36.82.7:4909 , piece_id=D64TFK7B6VMMTEBZPMU7ASYJ4QTCY344FAYBLHVP3BCRA3HN4EJQ, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.323460808 (190h8m32.054728458s) [noise=true , address=77.90.31.11:4554 , piece_id=VFJ44B73MIPGA6QLKRQ757K4GYQUFNUEFXGLYU25XZ3AMCQJP5LA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.323888371 (190h8m32.055156021s) [noise=true , address=45.91.116.8:4048 , piece_id=KUQUBG7YSCMA2ATH5PALBPFPAC4FMRTA2DIJXGOZLBGGY3YA67VA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.323974713 (190h8m32.055242363s) [noise=true , address=104.36.82.10:4077 , piece_id=RSHCC3DB24JBZMC6ZAMBMQFIDLVXBPDXWDDWKD3HYHU2ULWA7YMA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.324294123 (190h8m32.055561774s) [noise=true , address=45.91.116.5:4299 , piece_id=KH7BTGUQ3PLYH4MFBUWMEB7OUCIHMWF3BWU25YZGAIOBDND4MQ6A, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.324450587 (190h8m32.055718237s) [noise=true , address=77.90.31.11:4553 , piece_id=BZAFEMA4D5QHOBCJ2AEW6535ISDRDW3R2ACZSCPGGMNOO6YWF2MA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.324668918 (190h8m32.055936568s) [noise=true , address=45.91.116.7:4009 , piece_id=AWEDCB7TP6UPT7IJQ47OFN5MSBL36ZLSOWLLYILYB2TSQYIXB6HQ, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.324821434 (190h8m32.056089095s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BU67YMWAUMELTYN65SBZI7I3PIGVZJT2CJHRIGT25BRXEMIGPA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.325058770 (190h8m32.05632642s) [noise=true , address=45.91.116.12:4226 , piece_id=UT2QTXMT5ZX7SSKV2JHSZODZAGITWN3OW5A65KPLC2JG5GZZJ2UA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.325459874 (190h8m32.056727534s) [noise=true , address=104.36.82.4:4803 , piece_id=H3JRPUVZRZ6XAEYEGFVA5T2XL4VIUNLNQHMN5RLX2CI5FDGVEVAA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.325816795 (190h8m32.057084454s) [noise=true , address=45.91.116.4:4938 , piece_id=SKCZ7WHO4TWV77E7E7GJCXQE4V7USBOZWUTZX5ECXHVAICARZQPQ, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.326166252 (190h8m32.057433902s) [noise=true , address=104.36.82.6:4874 , piece_id=OUSZWQFEJSN7LM6HPY7DHOLMPTSMRYZM6D3P7D4ANIENG6WCKHYQ, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.326550834 (190h8m32.057818485s) [noise=true , address=104.36.82.8:4969 , piece_id=PZ33QUUZHCG22NBCOCVFKODLP6QX3NAYAGVT3BWYL6IYBSYVPU5Q, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.326900521 (190h8m32.058168181s) [noise=true , address=77.90.31.10:4519 , piece_id=B5RHOSIKTOHE6B74PNUKMXA2YO2ULEA2KK7M6NGGBUHO7Z7GFRLQ, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.327242344 (190h8m32.058509994s) [noise=true , address=77.90.31.13:4628 , piece_id=SPMZJTZTIEEX5RL2VI3Z2RPI3727AGYUI2Y4F3PAJZ2WUCHVHCNA, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.327611458 (190h8m32.058879107s) [noise=true , address=45.91.116.4:4937 , piece_id=P52HU6PKECIQ6GQIZOADSUHFO3TF5RUPV6W4REJA45H76NPF4LCQ, seg_pos={0 0}, upload=197851]: Uploading piece... Aug 16 15:34:51.434931608 (190h8m32.166199268s) [upload=197851]: Next returned nil segment Aug 16 15:34:51.434946095 (190h8m32.166213745s) [upload=197851]: Waiting for error group managing segments... Aug 16 15:34:51.445377046 (190h8m32.176644697s) [noise=true , address=104.36.82.10:4077 , piece_id=RSHCC3DB24JBZMC6ZAMBMQFIDLVXBPDXWDDWKD3HYHU2ULWA7YMA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.448398894 (190h8m32.179666545s) [noise=true , address=45.91.116.4:4937 , piece_id=P52HU6PKECIQ6GQIZOADSUHFO3TF5RUPV6W4REJA45H76NPF4LCQ, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.448853860 (190h8m32.18012151s) [noise=true , address=104.36.82.7:4909 , piece_id=D64TFK7B6VMMTEBZPMU7ASYJ4QTCY344FAYBLHVP3BCRA3HN4EJQ, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.449614199 (190h8m32.180881859s) [noise=true , address=104.36.82.6:4874 , piece_id=OUSZWQFEJSN7LM6HPY7DHOLMPTSMRYZM6D3P7D4ANIENG6WCKHYQ, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.449766164 (190h8m32.181033814s) [noise=true , address=45.91.116.12:4226 , piece_id=UT2QTXMT5ZX7SSKV2JHSZODZAGITWN3OW5A65KPLC2JG5GZZJ2UA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.450278567 (190h8m32.181546217s) [noise=true , address=77.90.31.10:4519 , piece_id=B5RHOSIKTOHE6B74PNUKMXA2YO2ULEA2KK7M6NGGBUHO7Z7GFRLQ, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.453705217 (190h8m32.184972867s) [noise=true , address=45.91.116.8:4048 , piece_id=KUQUBG7YSCMA2ATH5PALBPFPAC4FMRTA2DIJXGOZLBGGY3YA67VA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.462936393 (190h8m32.194204053s) [noise=true , address=45.91.116.4:4938 , piece_id=SKCZ7WHO4TWV77E7E7GJCXQE4V7USBOZWUTZX5ECXHVAICARZQPQ, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.464105601 (190h8m32.195373251s) [noise=true , address=45.91.116.5:4299 , piece_id=KH7BTGUQ3PLYH4MFBUWMEB7OUCIHMWF3BWU25YZGAIOBDND4MQ6A, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.468138338 (190h8m32.199405988s) [noise=true , address=45.91.116.7:4009 , piece_id=AWEDCB7TP6UPT7IJQ47OFN5MSBL36ZLSOWLLYILYB2TSQYIXB6HQ, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:34:51.787977685 (190h8m32.519245335s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BU67YMWAUMELTYN65SBZI7I3PIGVZJT2CJHRIGT25BRXEMIGPA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: <nil> Aug 16 15:35:21.279797711 (190h9m2.011065341s) [noise=true , address=77.90.31.13:4628 , piece_id=SPMZJTZTIEEX5RL2VI3Z2RPI3727AGYUI2Y4F3PAJZ2WUCHVHCNA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 5.161.89.252:16996->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:21.279817338 (190h9m2.011084957s) [noise=true , address=77.90.31.11:4554 , piece_id=VFJ44B73MIPGA6QLKRQ757K4GYQUFNUEFXGLYU25XZ3AMCQJP5LA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:21.279936181 (190h9m2.011203812s) [noise=true , address=77.90.31.11:4553 , piece_id=BZAFEMA4D5QHOBCJ2AEW6535ISDRDW3R2ACZSCPGGMNOO6YWF2MA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 5.161.89.252:61252->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:21.279959826 (190h9m2.011227446s) [noise=true , address=104.36.82.4:4803 , piece_id=H3JRPUVZRZ6XAEYEGFVA5T2XL4VIUNLNQHMN5RLX2CI5FDGVEVAA, seg_pos={0 0}, upload=197851]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 5.161.89.252:30432->104.36.82.4:4803: use of closed network connection; write tcp 5.161.89.252:30432->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:30432->104.36.82.4:4803: use of closed network connection Aug 16 15:35:21.279988079 (190h9m2.011255699s) [noise=true , address=104.36.82.8:4969 , piece_id=PZ33QUUZHCG22NBCOCVFKODLP6QX3NAYAGVT3BWYL6IYBSYVPU5Q, seg_pos={0 0}, upload=197851]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 5.161.89.252:33480->104.36.82.8:4969: use of closed network connection; write tcp 5.161.89.252:33480->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33480->104.36.82.8:4969: use of closed network connection Aug 16 15:35:21.280010601 (190h9m2.011278221s) [seg_pos={0 0}, upload=197851]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 16 15:35:21.295132207 (190h9m2.026399837s) [upload=197851]: Done uploading segments... Aug 16 15:35:21.295150391 (190h9m2.026418021s) [upload=197851]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.017110597s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X...................................X...X...................] duration: 186.720256ms FAIL (2025-08-16 15:34:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 186.735685ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X...................................X...X..................] duration: 3.470686433s FAIL (2025-08-16 15:33:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 3.470707683s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X..............X.................XX.........X.........] duration: 10.949645939s FAIL (2025-08-16 15:33:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 10.949664914s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X..............X.................XX.........X........] duration: 32.188622832s FAIL (2025-08-16 15:33:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 32.188650505s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X...................................X...X.................] duration: 2.636204689s FAIL (2025-08-16 15:32:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 2.63623183s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X..............X.................XX.........X.......] duration: 11.816088972s FAIL (2025-08-16 15:31:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 11.816108208s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [X..X...................................X...X................] duration: 9.376875396s FAIL (2025-08-16 15:31:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 9.376894482s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X..............X.................XX.........X......] duration: 23.458405449s FAIL (2025-08-16 15:31:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 23.458430806s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X..............X.................XX.........X.....] duration: 25.273881394s FAIL (2025-08-16 15:30:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 25.273904127s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.X..X...................................X...X...............] duration: 1.219002661s FAIL (2025-08-16 15:30:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 1.219030383s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X..............X.................XX.........X....] duration: 33.083500551s FAIL (2025-08-16 15:30:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 33.083517443s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..X..X...................................X...X..............] duration: 4.736961381s FAIL (2025-08-16 15:29:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 4.736991998s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X..............X.................XX.........X...] duration: 11.510814467s FAIL (2025-08-16 15:29:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== Aug 16 15:27:15.681967897 (190h0m56.413235527s) [upload=197717]: Starting upload Aug 16 15:27:15.682013522 (190h0m56.413281152s) [upload=197717]: Done starting upload Aug 16 15:27:15.682312493 (190h0m56.413580144s) [upload=197717]: Uploading segments... Aug 16 15:27:15.682390551 (190h0m56.413658202s) [upload=197717]: Got next segment. Inline: false Aug 16 15:27:15.682455282 (190h0m56.413722933s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:27:15.682461705 (190h0m56.413729355s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:27:17.443151844 (190h0m58.174419474s) [seg_pos={0 0}, upload=197717]: Begin upload segment... Aug 16 15:27:17.443186458 (190h0m58.174454088s) [seg_pos={0 0}, upload=197717]: Done begin upload segment. Aug 16 15:27:17.443211726 (190h0m58.174479356s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443237484 (190h0m58.174505114s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443515807 (190h0m58.174783437s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443588945 (190h0m58.174856575s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443742443 (190h0m58.175010073s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443879550 (190h0m58.17514718s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.443985961 (190h0m58.175253581s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444151302 (190h0m58.175418922s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444198621 (190h0m58.175466251s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444407292 (190h0m58.175674922s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444440385 (190h0m58.175708015s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444643326 (190h0m58.175910945s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444705433 (190h0m58.175973053s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444851387 (190h0m58.176119017s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.444964971 (190h0m58.176232591s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445066612 (190h0m58.176334242s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445279532 (190h0m58.176547162s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.445297526 (190h0m58.176565146s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Uploading piece... Aug 16 15:27:17.521480031 (190h0m58.252747661s) [upload=197717]: Next returned nil segment Aug 16 15:27:17.521495851 (190h0m58.252763471s) [upload=197717]: Waiting for error group managing segments... Aug 16 15:27:17.540752099 (190h0m58.272019719s) [noise=true , address=104.36.82.9:4025 , piece_id=Q2WBX2GPIY2Z6RGZJNYMOHYSZID4MZPMQXKEXXD7ZXKY54LUEBNA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.582955469 (190h0m58.3142231s) [noise=true , address=77.90.31.13:4653 , piece_id=BNUR3B23E3I2SNIQX2SQNX5427GJO3FI5HWAVEJHC3ZYSAXAWW2Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.596210569 (190h0m58.327478199s) [noise=true , address=104.36.82.34:4556 , piece_id=UKTLU5SB2IENGDQXHJ2SJULJBRILI4722UQGFBOOWLIB3PS6HYUQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.709271979 (190h0m58.44053963s) [noise=true , address=45.91.116.7:4017 , piece_id=NIPGYYXGAB6R6F3OPLWANKNNP76RUNDQ7O3VEN342K2REEQWH2IQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:17.796815115 (190h0m58.528082765s) [noise=true , address=79.127.220.101:4061 , piece_id=5MDL55V5KPPSKJRNREF332GO2JSJWY2VYV7SXJLDLF23ABECKYQA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.047385441 (190h0m58.778653071s) [noise=true , address=104.36.82.8:4989 , piece_id=7A4PL4WQM5SPT6Y7JAHRRAJWK6EFPNJJRRUPNXKWLVALL6DYJACQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.086472674 (190h0m58.817740315s) [noise=true , address=104.36.82.5:4864 , piece_id=WHSKYVVHESYBVLXOAD74GAFERDDRKSRGYYZSZPDJ4PXLPRKSIQYA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.137143568 (190h0m58.868411219s) [noise=true , address=79.127.201.215:4343 , piece_id=VZ6DEMP2VPEC3FYA5SYCVUNPSHYTSRTMW2XYLVIISJIXQM2BKJJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.141620952 (190h0m58.872888603s) [noise=true , address=104.36.82.60:4337 , piece_id=TSFZI537VZT2FZBOCS4MOOOJ76TRDCVWHOLMR5KSRBW55O6AZCJA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.147556878 (190h0m58.878824527s) [noise=true , address=77.90.31.16:4736 , piece_id=DGR6CRPNVPLRGI4LJXR66DQSTISAFKDAXE745WT3LLWE22N24IXA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.629103124 (190h0m59.360370754s) [noise=true , address=77.90.31.15:4704 , piece_id=J57XITN3RJ4T4A3JJGX77AJS6ZX35RSVXKHJXVHABRY3YRD65MSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.775361362 (190h0m59.506628981s) [noise=true , address=104.36.82.44:4886 , piece_id=2HVSI5U2KJMXU7T6W4MR64XZW357D37YILVLVIEJFSOSXTTK3QVA, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:18.836883656 (190h0m59.568151276s) [noise=true , address=104.36.82.3:4780 , piece_id=R225DVYC7YPAB7ZG44C62YBMVZFMMASBGC5W2OIG6TBOOCOCNOAQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096066596 (190h0m59.827334216s) [noise=true , address=79.127.201.214:4320 , piece_id=VJGXYXHGXUPPSQCL6USWRQPZIVRMABEYXIC7ZOFEM6US43RLFLPQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: <nil> Aug 16 15:27:19.096085662 (190h0m59.827353282s) [seg_pos={0 0}, upload=197717]: Segment reached optimal threshold of 14 pieces. Aug 16 15:27:19.096196951 (190h0m59.827464561s) [noise=true , address=66.23.193.175:4554 , piece_id=24JXRSMOTB4SKYQXCTY4M4ZCQUPR5B2X7VSJCJKR2FULLWKUQL5A, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1CraGuuckkbKe5PDLUvwF5G2ATUbgdR5g192YjkZgmP3JpqgXi): protocol: expected piece hash; context canceled; EOF Aug 16 15:27:19.096311055 (190h0m59.827578676s) [noise=true , address=77.90.31.19:4848 , piece_id=ONRM43N5E2QFMWHUOGV2LHIKMODL7UHAFJQS22AU7V5DAIDIP23Q, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD): protocol: write tcp 5.161.89.252:35648->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:27:19.096362723 (190h0m59.827630342s) [noise=true , address=104.36.82.36:4634 , piece_id=CURAJREZ575WSRXYQYV3FUY2E4IG5TM5OCZSAWIWE2UDYNFIKVSQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5): protocol: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:17176->104.36.82.36:4634: use of closed network connection Aug 16 15:27:19.096373433 (190h0m59.827641042s) [noise=true , address=104.36.82.39:4730 , piece_id=XR62A4LQRVEID4UWLNOFIPJYLUY6WKILHR4DMU6ONX6G47RRW3YQ, seg_pos={0 0}, upload=197717]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV): protocol: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41534->104.36.82.39:4730: use of closed network connection Aug 16 15:27:19.096386196 (190h0m59.827653817s) [seg_pos={0 0}, upload=197717]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 15:27:19.096461368 (190h0m59.827728988s) [upload=197717]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:27:19.096464924 (190h0m59.827732534s) [upload=197717]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:27:30.325241966 (190h1m11.056509616s) [upload=197717]: Done uploading segments... Aug 16 15:27:30.325260130 (190h1m11.05652779s) [upload=197717]: Upload finished. err: <nil> Duration: 11.510840337s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...X..X...................................X...X.............] duration: 3.403917165s FAIL (2025-08-16 15:29:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 3.403945137s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)