Split tunnel for Per-App VPN not working

I have built a VPN app using Packet Tunnel Provider on iOS. The VPN profiles are configured through an MDM server. The VPN can be used in a Per-App VPN configuration mode or device level mode.

I am facing an issue with the split tunnel in per-app VPN configuration. I set up a couple of routes for the Per-app VPN to split the traffic over the tunnel, but all the traffic go through the tunnel disregarding of the route setting.

When the same routes are configured and the VPN is running in device level mode, the route configuration is respected and only traffic destined to the specified networks goes through the tunnel.

The question I have is: Are the split tunnel routes not respected in per-app VPN configuration mode?

Below is the code snippet that I am using for configuring VPN setting.

//Set server address and assigned ip addresses
NEPacketTunnelNetworkSettings *setting = [[NEPacketTunnelNetworkSettings alloc] initWithTunnelRemoteAddress:remote_server_addr];
setting.IPv4Settings = [[NEIPv4Settings alloc] initWithAddresses:tunnel_addresses subnetMasks:netmasks];


//Add routes
if(routes.count > 0)
{
    NSMutableArray <NEIPv4Route *> * includeRoutes = [[NSMutableArray alloc]init];
    for (int i = 0; i < routes.count; ++i)
    {
        NEIPv4Route* route = [[NEIPv4Route alloc] initWithDestinationAddress:routes[i].addr subnetMask: routes[i].netmask];
        if (routes[i].gateway.length() > 0) {
            route.gatewayAddress = routes[i].gateway;
        }
        [includeRoutes addObject:route];
    }
    setting.IPv4Settings.includedRoutes = includeRoutes;
}
else
{
    setting.IPv4Settings.includedRoutes = @[[NEIPv4Route defaultRoute]];
}

//Set DNS server
setting.DNSSettings = [[NEDNSSettings alloc]initWithServers:DNSServers];
setting.DNSSettings.searchDomains = nil;
setting.DNSSettings.matchDomains = @[@""]

setting.tunnelOverheadBytes = @(1500);

[tun_provider setTunnelNetworkSettings:setting completionHandler:^(NSError * _Nullable error) {
    //handle result
}];