1

Problem is on multiple hosts but lets chase one.

Host 10.66.100.200, 255.255.255.0, gw is 10.66.100.250

guest (on the same server) :slight_smile: 10.66.100.210 gw 10.66.100.250,

DNS 1 internal address, and open dns 208.67.220.220

The guest had a static address (per above) and since nothing was working I tried dhcp as a test.

DHCP did give the guest an address 10.66.100.1, gw 10.66.100.250 mask of 24.

Guest to host (or the other way) and ping fails (fw is OFF on both),

Host can get ping gateway of 10.66.100.250, the guest can not… If I ping host to guest I get destination host unreachable.

The guest and host share the same NIC. This used to work. There is a firepower 2100 FW in the mix but I thought it wasn’t in play from host-guest or guest-host transmission.

If the FW is in play in this problem any suggestions on what I have that team look at. They’ve (including a cisco level 1 person) have "modified the rules enough times all bets are off. I’m looking to see if I have wireshark installed on any of the guests so I can get a clue in the guest as to what the HECK is going on. The fact that I got dhcp indicates some communication.

Anything else I can do on the hyper-v side. (I have already deleted and re-added guest NIC and guest virtual switch and recreated them to no help) I have rebooted the host.)

server 2019, hyper-v.

6 Spice ups
2

Your problem is likely right here.

You should never use internal and external DNS. Internal system should ONLY use internal DNS. Your DC can use root hints or forwarders, but it’s DNS settings should only ever be itself (if you have one) or another DC then itself if you have multiples.

Clients should only ever use the DC for DNS

Your GW should be whatever it handling your routing.

If DHCP is handing out a different GW to that of what you want it to be, change it in the DHCP scope.

1 Spice up
3

Well sort of. This was more for diagnostic sake at this point. If I can’t ping my gateway I can’t get to my dns server inside or outside. The actual question is more why my guests can’t communicate then DNS. I’m testing more things at a command line and pinging the gateway for the vlan for the guest. It works from the host and fails at the guest. (pinging the vlan gateway)

So DNS is not the major problem the lack of overall connectivity is. The fact that I get a dhcp address on the guest (which since it’s a DC is ONLY for testing right now after I changed it off of static) but I can’t ping or communicate with anything else… suggests the firewall but I’d like to confirm it’s not some Hyper-V setting that I’ve borked. (I don’t think so LOL but it’s not working so somethings busted) I don’t even get a name resolution for google attempting a ping so that guest is not getting out correctly.

Thanks Rod, any thoughts on the connectivity issue? (not just dns)

Rod-IT wrote:

Your problem is likely right here.

You should never use internal and external DNS. Internal system should ONLY use internal DNS. Your DC can use root hints or forwarders, but it’s DNS settings should only ever be itself (if you have one) or another DC then itself if you have multiples.

Clients should only ever use the DC for DNS

Your GW should be whatever it handling your routing.

If DHCP is handing out a different GW to that of what you want it to be, change it in the DHCP scope.

4

You’ve mentioned VLANs, but in your first post, both devices are on the same subnet.

Where do VLANs come in to play?

Are the guests on the same vSwitch?

5

LOL yes they’re on the same switch. It’s rather fundamental. The host can’t communicate with guest and vice versa. There are other vlans but the host is on one nic, the virtual switch is sharing the same nic.(all on the same vlan 10.33.66.0) and it all worked before the other team started fussing with the firepower firewall.

I don’t know if it’s the FW as the root issue but I lean heavily in that direction. I still need to see if I have wireshark on one of the guests. I just want to rule out hyper-v and “me” as the source of the issue. I’m easy to fix but if I have to persuade the fw team… uh that’s harder

6

There are so many questions on so many levels ??

For starters if you are using server 2019 with hyper-v role, how many IPs does the host have (if more than 1 NIC, are they teamed) ?

Why is there a need to “share” NIC between VM & host ? Why not use a vSwitch ?

Where are the following (if any)

  • DCs
  • DHCP server
  • DNS servers (Internal & external)

If you use external DNS…then if the VM broadcasts for IP address (yelling…“DHCP server gimme IP address” )…who will answer if that broadcast goes out to Internet ?

I am not asking a Cisco L1 person but I did ask my 15 yr old dotter to read your question and that “yelling” example was given by her but I think she cheated a little as I think she read @rod-it . answer already…

7

I think you have to back up and explain the addressing again.

You said the host and guest were on 10.66.100/24. Now the host and virtual switch are on 10.33.66/(/24 again presumably?)

The firewall you mention… that’s an external device right? Shouldn’t the host and guest be able to talk to one another even if you disconnect the LAN cable? They will talk internally, not through an external device.

Are you sure the guest is getting an address from DHCP and not just re-applying an address it had already? Perhaps reset the TCP/IP stack on the guest.

8

from [elevated] command prompt:

C:>netsh int ip reset

C:>netsh winsock reset

C:>shutdown /r /t 0 (number of seconds delay)

9

What kind of virtual switch is the guest connected to? It needs to be external.

1 Spice up
10

Yes it’s external vswitch As noted this had worked previously. I had the guest as static previously (which since it’s a dc is what it’s supposed to be) playing around with dhcp was just an attempt to see what communication if any was occuring.

====================

" I think you have to back up and explain the addressing again. (I’d agree with that LOL)

You said the host and guest were on 10.66.100/24. Now the host and virtual switch are on 10.33.66/(/24 again presumably?)

YEP I’ve munged the address so I want to make sure that munging doesn’t introduce a typo by me. They are on the same subnet.

The firewall you mention… that’s an external device right? Shouldn’t the host and guest be able to talk to one another even if you disconnect the LAN cable? They will talk internally, not through an external device. Are you sure the guest is getting an address from DHCP and not just re-applying an address it had already? Perhaps reset the TCP/IP stack on the guest.

===================

10.66.100.200 for host and 10.66.100.1 was what dhcp was handing the guest. the guest was static at 10.66.210.210 previously and had the same issues. The dhcp is diagnostic only so (shall we say) don’t go down the rabbit hole of dc’s should be static… yep I know that :slight_smile:

I’ll reset the stack but I don’t think it’s getting the same address (but what the heck its broken anyway) I’ve rebooted the guest enough times at this point., what’s one more. FW is handling dhcp not the local DC (which was not my call and I can’t change it)

Shouldn’t the host and guest be able to talk to one another even if you disconnect the LAN cable? They will talk internally, not through an external device.

Yep that’s my question too., hence the boy this is odd post. As I note I’m trying to rule out hyper-v and blame it on the FW changes (that I didn’t do) BUT I’m a variable and I might have screwed up the hyper-v guest config.

11

Your replies are hard to read.

When replying to someone’s comment, please click reply under their post, then quote post to the right, only retain the pieces you need.

As far as blaming the FW, I can only see this being true if this is the GW and something has changed relating to your host or physical NIC ports - perhaps the VLAN this is on has been changed or no longer mapped to the NICs you are using.

12

Sorry the interface is sometimes less than helpful. I keep looking for the color section and trying make it easy to read. The biggest puzzler is the guest and host networking should not even hit the lan (at least they don’t on vmware). The point one guy made of "hey if the nic’s disconnected the guest/host should still communicate is why I’m so baffled by the whole thing.

There were all sorts of changes made to the FW of which I’m not all aware of watching the Cisco rep and the Fw team make them.

Rod, as noted, I’m trying to rule out a misconfig of guest/host in Hyper-v but per the point above… when guest and host are on the same device and can’t communicate that’s really puzzling.

13

Can you check arp (arp -a) tables on host and guest? Even if there is a firewall, valid arp entries should still get generated. If there’s no arp entries for something you try to ping, the problem is at L2, L1, or network stack is completely messed up.

2 Spice ups
14

The above will be true if everything is contained within the subnet and all local to the host. If DNS, DHCP or your GW are outside of the host, then access to that external source will be needed for parts.

It’s hard to diagnose though because you’ve mentioned two networks, firewall changes and you’ve changed DNS and DHCP settings. Your issue could be any combination of things.

15

The above will be true if everything is contained within the subnet and all local to the host. If DNS, DHCP or your GW are outside of the host, then access to that external source will be needed for parts.

It’s hard to diagnose though because you’ve mentioned two networks, firewall changes and you’ve changed DNS and DHCP settings. Your issue could be any combination of things.

Now that’s brilliant. (seriously) since the GW is outside of the host. I may just change the guest/host IP to something local and then see if the 2 little suckers can talk to each other. In terms of the FW changes, yep a huge variable. I’d discount dns/dhcp since my original post was listing where I ended up after changes… it started broken before I started mucking in the soup. I’m testing at a command line for nearly everything.

Let me find wireshark and find out wth is going on. (always a good time) I do wish I controlled that darned fw

16

If you have two machines on the same subnet with no gateway, you should be able to connect to each using IPs.

You could also change the vSwitch to internal and re-test, but given how many changes have already been made, both by you and from the Firewall perspective, at best we can only guess.

Can either device ping the gateway, if they can I would doubt this is the issue.

17

Create an additional internal virtual switch between the host and the guest (no DHCP), manually assign 192.168.x.x or 172.16.x.x subnet IPs and mask to newly created virtual network adapters and check if that connection works. At least that would help you determine the problem is somewhere starting from the external virtual switch and further in the network and not the host/guest themselves.

18

where is the dhcp server? (also a vm on same host? or a physical on network etc?) The fact that the vm gets an ip from dhcp server means it can communicate with the dhcp server.

From the ip addressing the host and vm should communicate, and the vm should be able to ping the gateway.

The error " destination host unreachable" indicates a networking problem - no response to arp can give this error. Check arp table likle Kevin advised. Make sure the L2 path is correct - ‘external v-switch’ no vlan tag applied etc.

19

So (as this quest continued) I did the following:

  1. Removed all 3 vlans from the firewall and put them on 3 unmanaged switches. (that removes the FW from the equation)

  2. Yes removing the FW creates some issues but this was testing.

  3. On each vlan tested some static devices from guest-host etc.

  4. When I still had issues (per the above)… well one issue now is not the FW

5. Said-: lets start over and I (as you all aptly pointed out… way too many changes… where the heck are we LOL ROD-IT great points… hope to heck you can read this)

a. deleted all the virtual switches on the 2 hosts

b. created a private network on both hosts (thank you person above VERY good suggestion)

c. private network worked

d. re-labeled the local NIC’s, made sure I could ping a static device on the same (now) standalone subnet

e. disabled any NIC’s that weren’t doing anything to keep them OFF my radar (verified via ILO)

f. recreated the virtual switches from scratch

g. re-tested host to guest… shazam IT WORKED. (conclusion some change by yours truly whacked something)

h.Plugged the 3 unmanaged switches back into their respective FW ports

i. re-testing 2 of three vlans worked, the vlan with outside dmz access still has an issue so now I concentrate on that

So-what did I screw up. NO idea the switch configs looked the same but obviously something wasn’t. Starting over, isolating out the FW was very helpful in ruling it out.

The Meraki is still way screwed up and won’t talk to the internet. THAT I deal with Meraki on.