I think this is an SDK bug. The workaround is either to add a specific SourceLink provider package reference (which may be redundant and may even be problematic or impossible) or to add a <SourceRoot Include="$(SolutionDir)" /> item or similar. The actual directory does not matter so long as it contains all source files not already covered by a SourceRoot item.

Without the workaround, you can see absolute paths leaking from the CI image, and NuGet Package Explorer also flags the resulting assembly as failing the determinism check:

Using the workaround, determinism is achieved:

If no specific SourceLink provider package is referenced in a project, the SDK should add a SourceRoot directory by default containing all source files. This should work the same way whether or not the project is even contained in a source-control repository of any kind. Alternatively, source files outside the solution directory should not be mapped and should cause build warnings when ContinuousIntegrationBuild=true.