Ever since Apple were forced to ditch Samba due to their changing the licensing to GPL3, we’ve been stuck with a second-rate SMB implementation on OS X and Server called smbx.
As it turns out, there are numerous issues with smbx that may or may not end up getting fixed that were pretty much solved problems in Samba. That, and there’s a lot more support out there for Samba across all of the platforms it runs on than there is for smbx.
Oh well, we must make do with what we’ve got…
Windows 10, it seems, has issues connecting to smbx on recent versions of OS X or macOS – something to do with the LanMan compatibility level – i.e. the version of SMB/CIFS that Windows speaks. What it looks like is happening is that Windows is trying to use the older LM or NTLM authentication whereas the Mac wants the newer, and more secure, NTLMv2 authentication method to be used. The result however is that you’re told you have an incorrect username or password.
I’m still looking for a fix on the Mac server side, however I have found a fix that works when applied to the Windows client.
On Windows, open Regedit and go to:
Find the key for LmCompatibilityLevel and change it to 3 (from the default value of 1)
Quit out of Regedit and be amazed that you can now connect to the Mac server.
There is more info over on TechNet if you want to know exactly what this key does.
I haven’t tested to see if denying SMB3 connections at the server end makes any difference…