anav, thanks so much for the incredible feedback! I have responded to your detailed comments below. Unfortunately I still don't have it working (see my last comments). I think I may try to just mangle it since VRF is not quite ready yet. I assume at this point, it is the state of VRF that is my problem and not my config.
(1) Just need clarification on the allowed addresses.............. I am assuming from the below that you want VLAN users to be able to access on VPS or beyond, the two subnets shown and NOT internet (10.0.1.0/24,10.0.3.0/24)?? Further, the wireguard vlan will not have local internet access nor access to other vlans on the local router right?
100% Correct. VLAN 100 should only get to the internet through the VPS via wireguard 1
(2) The wireguard address should be properly formatted see below:
add address=10.255.255.2/24 comment="wireguard1 VPN to VPS" interface=wireguard1 network=10.255.255.0
Thanks, fixed!
(3) I think your firewall rules are way over complicated,.......... why all the jump rules, especially for INPUT chain.
That is the reason to have interface lists......... simplify simplfify
All the ones that need DNS service - then you have one input chain rule
All the ones that need NTP service - then you have one input chain rule.I do like the fact that you use drop all, so forget about all the other nonsense.
and add back in after the input chain established related
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
Then put your very nice admin rule (even though tis duplicated - a symptom of a bloated out of control firewall).
add action=accept chain=input comment="VL10_MGMT - ANTI-LOCKOUT - WINBOX" dst-address=192.168.10.1 dst-port=8291 in-interface-list=MGMT protocol=tcpExcept I would never publish here my actual dst-port, and depending on who else was on MGMT, would consider an additional firewall address list called "authorized" for example.
Ok, done! I really did not like how duplicative my firewall got, the smart use of interface lists is great! I will have to change my MGMT port after I am done troubleshooting as well. Thanks for pointing out the security issue.
Also, there is absolutely no need for a separate drop broadcast rule on the input chain, that is covered in the next rule the drop all rule, making it redundant.
Yes I agree thats true, but I keep it there so that I can de-clutter my drop all logs from all the broadcast hits. If there is a better way, I am all ears.
(4) This needs to be removed as the MT is a client NOT a server and does not require an input chain rule at all???
add action=jump chain=input comment="wireguard1-input: JUMP -- CAUTION!: no dst address filter" jump-target=wireguard1-input src-address-list=wireguard1-share
add action=drop chain=wireguard1-input comment="wireguard1-input: DROPALL" log=yes log-prefix="wireguard1-input: DROPALL"
Concur, I deleted it.
(5) Instead of cluttering the MT device with all these outgoing vlan100 forward chain rules.. you could put them on the VPS end........ I suppose it doesnt matter either way......... Looks like the wireguard 1 access to the VPS is very limited to certain ports etc......... For me it might make more sense to limit to specific IPs either at the MT router or at VPS end as you give whole subnet access on the allowed IPs, which is the way I would do it as well.
Thanks for the insight here. I think I am more comfortable being more stringent on what I percieve as the network "edge" vice moving it to the VPS.
(6) I think you are missing an interface on this rule?? I am assuming the jump chain is to control
either users on the VPS accessing the VLAN100, or users on VLAN100 accessing services on the VPS. Something I found too difficult to figure out LOL.
add action=jump chain=forward comment="wireguard1-forward: JUMP" jump-target=wireguard1-forward src-address-list=wireguard1-share
So this used to use the wireguard1 intrerface as the source, but it would not pick up traffic from the VPS subnets (10.0.1.0/24, 10.0.3.0/24, 10.0.4.0/24) so I swiched to using the address list.
(7) WHY, you have a drop rule at the end of the forward chain??
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
All you need is:
Ha good catch here.... thanks!
(8) Any rules such as this are redundant as well and take care of by the drop rule!!
add action=drop chain=VL100-input comment="VL100-input: DROP intervlan traffic - ignore broadcast (taken care of down the chain - log spam guard)" dst-address-list=!VL100_Addresses dst-address-type=!broadcast log=yes log-prefix=\
"VL100-input: DROP intervlan traffic"
Agreed. Deleted.
(9) WHY ARE you MANGLING vlan 100 traffic, that was the purpose of using VRF, so that you didnt have to use mangle or route rules !!!
I did this when I started to think the VRF was broken. I have removed them now.
(10) Why are you source NATTING out wireguard one? First of all,
you control the VPS its not a third party VPN and thus for allowed IPs at the VPS just put in the subnet of the VL100 (10.0.4.0/24) ?? No sourcenat required.
If you did for any reason the format is simpler
add action=masquerade chain=srcnat disabled=yes out-interface=wireguard1Okay then I see the next rule......... is also not correct, the one above works just fine but again I see this rule too is disabled ?????
So perhaps you realized you do not have to source nat here after all LOL.
add action=src-nat chain=srcnat comment="defconf: masquerade" disabled=yes dst-address-list=!wireguard1-share log=yes log-prefix="SNAT-WG0: " src-address-list=VL100_Addresses to-addresses=10.255.255.2OKAY I see its disabled LOL
Yes I agree that SNAT is not required here.... I started just poking at things to see if I could get the route to work.... removed.
(11) You dont need any of your bogons and other crap rules of any chain, prerouting, raw etc........ garbage from youtube etc....................
If you really want to do bogons just do something like.
/ip route
add blackhole disabled=no dst-address=10.0.0.0/8
add blackhole disabled=no dst-address=172.16.0.0/12
add blackhole disabled=no dst-address=192.168.0.0/16
Yes my thinking has evolved on this, I don't think i need them either. I deleted the RAW table inputs.
(12) My understand is that VRF creates the necessary routes for subnets within the VRF and there is only one, so thats fine....... BUT not sure what this shows??
Did you manually enter them......
add disabled=no distance=1 dst-address=10.0.1.10/32 gateway=wireguard1@vrf_earthcloud pref-src="" routing-table=vrf_earthcloud scope=11 suppress-hw-offload=no target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.0.1.10@vrf_earthcloud routing-table=vrf_earthcloud scope=30 suppress-hw-offload=no target-scope=11
add disabled=no distance=1 dst-address=YYY.YYY.Y.YY/32 gateway=wireguard1@vrf_earthcloud pref-src="" routing-table=vrf_earthcloud scope=30 suppress-hw-offload=no target-scope=10
Yes I manually added these trying to get the default route to work.... They are gone now.
In any case there
The only two you need are as follows:
add dst-address=10.0.1.0/24 gwy=wireguard1@vrf_earthcloud table=vrf_earthcloud
add dst-address=10.0.3.0/24 gwy=wireguard1@vrf_earthcloud table=vrf_earthcloud
Yes I have these routes. I am not sure why the export did not include the routing table... weird.
How to get there from here.
Still not to hard, no mangling required./ip routing table
add name=vrf_earthcloud fib/ip route
add dst-address=10.0.1.0/24 gwy=wireguard1 table=vrf_earthcloud
add dst-address=10.0.3.0/24 gwy=wireguard1 table=vrf_earthcloud/routing rule add src-address=10.0.4.0/24 action=lookup-only-in-table table=vrf_earthcloud.
Ok so i added these rules (however, it would only accept them in the following format):
/ip route
add dst-address=10.0.3.0/24 gateway=wireguard1@vrf_earthcloud routing-table=vrf_earthcloud
It wanted the gateway to be @ the vrf even with the routing table specified it seems.
What is striking to me with this solution is that now I do not have a default route (0.0.0.0/0) on the VRF table. I would think i need that so it knows to route outside request through wireguard1
The rule is also interesting because the VLAN100 interface is already attached to the VRF: (But I added the rule anyway)
Code: Select all
/ip/vrf> printFlags: X - disabled; * - builtin 0 name="vrf_earthcloud" interfaces=wireguard1,VL100_WIREGUARD 1 * name="main" interfaces=all With the changes you suggested, Here is the ip table:
Code: Select all
/ip/route> printFlags: D - DYNAMIC; A - ACTIVE; c, s, d, y - COPYColumns: DST-ADDRESS, GATEWAY, DISTANCE# DST-ADDRESS GATEWAY DISTANCE DAd 0.0.0.0/0 208.102.16.1 1 DAc 10.0.2.2/32 wireguard2 0;;; Route for local laptop WG connection0 As 10.0.5.0/24 wireguard2 1 DAc 192.168.10.0/24 VL10_MGMT 0 DAc 192.168.20.0/24 VL20_TRUST 0 DAc 192.168.35.0/24 VL35_CELL 0 DAc 192.168.40.0/24 VL40_GUEST 0 DAc 192.168.50.0/24 VL50_CLEARNET 0 DAc 192.168.51.0/24 VL51_MEDIA 0 DAc 192.168.52.0/24 VL52_IOT 0 DAc 192.168.55.0/24 VL55_P2P 0 DAc 192.168.60.0/24 VL60_LAB 0 DAc XXX.XXX.XX.X/20 VL90_PASSUNTRUSTED 01 As 10.0.1.0/24 wireguard1@vrf_earthcloud 12 As 10.0.3.0/24 wireguard1@vrf_earthcloud 1 DAc 10.0.4.0/24 VL100_WIREGUARD@vrf_earthcloud 0 DAc 10.255.255.0/24 wireguard1@vrf_earthcloud 0Unfortunately even with these changes it appears I still cannot ping 8.8.8.8 via VLAN100
Code: Select all
/tool/traceroute vrf=vrf_earthcloud address=8.8.8.8 Columns: LOSS, SENT, LAST# LOSS SENT LAST 1 100% 1 timeout2 100% 1 timeout3 100% 1 timeout4 100% 1 timeout5 0% 1 0ms I don't see the pings hit the VPS with TCPDUMP at all.
Perhaps i just need to abandon the VRF and instead use mangle to route anything outside wireguard-1 share to the VPS via wireguard-1? I assume if this config "should" work it is simply because of the state of VRF and not an error in my config.
anav, Thanks so much for giving me such detailed feedback, it is incredibly appreciated on my end. Thanks!!!
for your reference, due to your suggestion my firewall now look like this 
(all RAW rules are gone):
Code: Select all
/ip/firewall/filter> printFlags: X - disabled, I - invalid; D - dynamic 0 D ;;; special dummy rule to show fasttrack counters chain=forward action=passthrough 1 ;;; defconf: accept established,related,untracked chain=forward action=accept connection-state=established,related,untracked log=no log-prefix="" 2 ;;; defconf: fasttrack chain=forward action=fasttrack-connection hw-offload=yes connection-state=established,related log=no log-prefix="" 3 ;;; defconf: accept all that matches IPSec policy chain=forward action=accept log=no log-prefix="" ipsec-policy=in,ipsec 4 ;;; defconf: accept established,related,untracked chain=input action=accept connection-state=established,related,untracked log=no log-prefix="" 5 ;;; defconf: drop invalid chain=input action=drop connection-state=invalid 6 ;;; defconf: accept ICMP chain=input action=accept protocol=icmp 7 ;;; VL10_MGMT - ANTI-LOCKOUT - WINBOX chain=input action=accept protocol=tcp dst-address=192.168.10.1 in-interface-list=MGMT dst-port=8291 log=no log-prefix="" 8 ;;; input: accept DNS chain=input action=accept protocol=udp in-interface-list=DNS_interfaces dst-port=53 log=no log-prefix="" 9 ;;; input: accept NTP chain=input action=accept protocol=udp in-interface-list=NTP_interfaces dst-port=123 log=no log-prefix="" 10 ;;; input: accept DHCP chain=input action=accept protocol=udp in-interface-list=DHCP_interfaces src-port=68 dst-port=67 log=no log-prefix="" 11 ;;; input: accept DNS requests from earthcloud on Wireguard1 chain=input action=accept protocol=udp src-address-list=wireguard1-earthcloud dst-port=53 log=no log-prefix="" 12 ;;; VL20-input: Allow wireguard2 access chain=input action=accept protocol=udp dst-address-type="" in-interface=VL20_TRUST dst-port=13231 log=no log-prefix="" 13 ;;; input: DROP BROADCAST [log spam guard] chain=input action=drop dst-address-type=broadcast log=no log-prefix="" 14 ;;; input: DROPALL chain=input action=drop log=yes log-prefix="input:DROPALL" 15 ;;; forward: Allow WAN access for specific interfaces chain=forward action=accept in-interface-list=WAN_ACCESS out-interface-list=WAN log=no log-prefix="" 16 X ;;; forward:WAN ACCESS TOGGLE - MGMT Net chain=forward action=accept in-interface-list=MGMT out-interface-list=WAN log=no log-prefix="" 17 ;;; VL55-forward: ALLOW ACCESS TO SPECIFIC VPN SERVER ADDRESS ONLY (DISABLE WAN ACCESS BELOW) -- NY#54 chain=forward action=accept dst-address=37.120.244.62 in-interface=VL55_P2P out-interface-list=WAN log=no log-prefix="" 18 X ;;; forward: ALLOW WAN ACCESS TOGGLE - VL55-P2P chain=forward action=accept in-interface=VL55_P2P out-interface-list=WAN log=no log-prefix="" 19 ;;; forward: SNAPCAST - Stream Port 1704 - VL52-IOT to SNAPCAST SERVER (VL20) chain=forward action=accept protocol=tcp dst-address=192.168.20.100 in-interface=VL52_IOT dst-port=1704 log=no log-prefix="" 20 ;;; forward: SNAPCAST - Stream Port 1705 - VL52-IOT to SNAPCAST SERVER (VL20) chain=forward action=accept protocol=tcp dst-address=192.168.20.100 in-interface=VL52_IOT dst-port=1705 log=no log-prefix="" 21 ;;; forward: ALLOW to wireguard1 from VL100 chain=forward action=accept dst-address-list=wireguard1-share in-interface=VL100_WIREGUARD log=no log-prefix="" 22 ;;; forward: ALLOW to wireguard1 from VL100 chain=forward action=accept dst-address-list=wireguard2-share in-interface=VL100_WIREGUARD log=no log-prefix="" 23 ;;; wireguard1-forward: JUMP chain=forward action=jump jump-target=wireguard1-forward src-address-list=wireguard1-share log=no log-prefix="" 24 ;;; wireguard2-forward: JUMP chain=forward action=jump jump-target=wireguard2-forward in-interface=wireguard2 log=no log-prefix="" 25 ;;; allow dst-nat chain=forward action=accept connection-nat-state=dstnat 26 ;;; defconf: drop bad forward IPs chain=forward action=drop src-address-list=no_forward_ipv4 27 ;;; defconf: drop bad forward IPs chain=forward action=drop dst-address-list=no_forward_ipv4 28 ;;; DROP anything form VL90 for fowarding - this is essentially WAN chain=forward action=drop in-interface=VL90_PASSUNTRUSTED log=no log-prefix="DROPALL" 29 ;;; DROP ALL ELSE chain=forward action=drop log=yes log-prefix="forward: DROPALL" 30 ;;; wireguard1-forward: Allow to VL100 - ICMP chain=wireguard1-forward action=accept protocol=icmp dst-address-list=wireguard1-share log=no log-prefix="" 31 ;;; wireguard1-forward: Allow to VL100 - SSH chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=22 log=no log-prefix="" 32 ;;; wireguard1-forward: Allow to VL100 - HTTP access chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=80 log=no log-prefix="" 33 ;;; wireguard1-forward: Allow to VL100 - HTTPS access chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=443 log=no log-prefix="" 34 X ;;; wireguard1-forward: Allow to VL100 - NFS chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=2049 log=no log-prefix="" 35 X ;;; wireguard1-forward: Allow to VL100 - Kerberosv5 - TCP chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=88 log=no log-prefix="" 36 X ;;; wireguard1-forward: Allow to VL100 - Kerberosv5 - UDP chain=wireguard1-forward action=accept protocol=udp dst-address-list=VL100_Addresses dst-port=88 log=no log-prefix="" 37 X ;;; wireguard1-forward: Allow to VL100 - Kerberosv5 - kadmin - UDP chain=wireguard1-forward action=accept protocol=udp dst-address-list=VL100_Addresses dst-port=749 log=no log-prefix="" 38 X ;;; wireguard1-forward: Allow to VL100 - Kerberosv5 - kadmin - TCP chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=749 log=no log-prefix="" 39 X ;;; wireguard1-forward: Allow to VL100 - iperf chain=wireguard1-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=5001 log=no log-prefix="" 40 ;;; wireguard1-forward: DROPALL chain=wireguard1-forward action=drop log=yes log-prefix="wireguard1-forward: DROPALL" 41 ;;; wireguard2-forward: Allow to VL100 - ICMP chain=wireguard2-forward action=accept protocol=icmp dst-address-list=wireguard2-share log=no log-prefix="" 42 ;;; wireguard2-forward: Allow to VL100 - SSH chain=wireguard2-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=22 log=no log-prefix="" 43 ;;; wireguard2-forward: Allow to VL100 - NFS chain=wireguard2-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=2049 log=no log-prefix="" 44 ;;; wireguard2-forward: Allow to VL100 - RPC for NFS chain=wireguard2-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=111 log=no log-prefix="" 45 ;;; wireguard2-forward: Allow to VL100 - Kerberosv5 - TCP chain=wireguard2-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=88 log=no log-prefix="" 46 ;;; wireguard2-forward: Allow to VL100 - Kerberosv5 - UDP chain=wireguard2-forward action=accept protocol=udp dst-address-list=VL100_Addresses dst-port=88 log=no log-prefix="" 47 X ;;; wireguard2-forward: Allow to VL100 - iperf chain=wireguard2-forward action=accept protocol=tcp dst-address-list=VL100_Addresses dst-port=5001 log=no log-prefix="" 48 ;;; wireguard2-forward: DROPALL chain=wireguard2-forward action=drop log=yes log-prefix="wireguard2-forward: DROPALL" 