SharePoint 2013 Send Email Workflow action failing for certain email addresses

We are having issues where a 2013 workflow send email action is not working for certain users.  Have tried a full profile sync from AD and checked work email etc is populated.

Looking at the requests to workflow server in fiddler shows the email address is being passed OK.  The error in the response to /fire/_vti_bin/client.svc/sp.utilities.utility.SendEmail is - 

{"error":{"code":"-2130242040, Microsoft.SharePoint.SPException","message":{"lang":"en-US","value":"The e-mail message cannot be sent. Make sure the e-mail has a valid recipient."},"innererror":{"message":"The e-mail message cannot be sent. Make sure the e-mail has a valid recipient.","type":"Microsoft.SharePoint.SPException","stacktrace":"   at Microsoft.SharePoint.Utilities.SPUtility.SendEmail_Client(EmailProperties properties)\r\n   at Microsoft.SharePoint.ServerStub.Utilities.SPUtilityServerStub.InvokeStaticMethod(String methodName, ClientValueCollection xmlargs, ProxyContext proxyContext, Boolean& isVoid)\r\n   at Microsoft.SharePoint.Client.ServerStub.InvokeStaticMethodWithMonitoredScope(String methodName, ClientValueCollection args, ProxyContext proxyContext, Boolean& isVoid)\r\n   at Microsoft.SharePoint.Client.Rest.RestRequestProcessor.GetObjectFromPathRoot(Boolean mainRequestPath, EdmParserNode node, Boolean resourceEndpoint, MethodInformation& methodInfo)\r\n   at Microsoft.SharePoint.Client.Rest.RestRequestProcessor.GetObjectFromPath(Boolean mainRequestPath, String path, String pathForErrorMessage)\r\n   at Microsoft.SharePoint.Client.Rest.RestRequestProcessor.Process()\r\n   at Microsoft.SharePoint.Client.Rest.RestRequestProcessor.ProcessRequest()\r\n   at Microsoft.SharePoint.Client.Rest.RestService.ProcessQuery(Stream inputStream, IList`1 pendingDisposableContainer)","internalexception":{"message":"The e-mail message cannot be sent. Make sure the e-mail has a valid recipient.","type":"Microsoft.SharePoint.Client.ClientServiceException","stacktrace":""}}}}

Looking in ULS viewer, the following error seems to be the root cause - 

ResolveAddressesForEmail : Resolved input 'xx@xx.xx' is not registered as the site collection user, ignored. Login xx

So it looks like the user is being resolved, but something else is wrong.  When trying to email the site owner, even they get this error.  Emails to most of the users are working, just a select few.

Any idea why this is happening?