One way you could do that (not the most optimal solution) is by saving your log into Codable struct, that could be converted to String JSON, and publishing it in OSLog stream with your specified subsystem. And then retrieve these logs with JSON decoding back into struct from your main target, where you want to add it on the UI, etc.
Post
Replies
Boosts
Views
Activity
It seems like the issue was also related to the way I am handling packets in DNSProxy as the packet comes in hex form, it was possible to compare the values of expected packet hex and the actual one. It was found that my code wasn't properly serializing the packets from structure because of the name pointers.
After it was resolved I was able to add immediate block with just returning proper packet for nxdomain. Even though I am still working on remediation, I think it should work similarly
Let's decouple this a bit. If you do not use the Content Filter and just use NEDNSProxyProvider, does it handle WebKit / Safari flows properly? Are you able to resolve these flows?
Yes, it works with the content filter on as well. Logs are showing proper flows handling from NEDNSProxyProvider.
Are you using NEDNSProxyProvider as a mechanism to block traffic? If so, what is your content filter doing?
I needed Content Filter to handle IP-connections or connections that don't have a hostname. At first the app didn't have Content Filter Providers, but then I soon realised that I needed it to handle traffic that's not intercepted by the NEDNSProxyProvider
The problem with this is that my NRDNSProxyProvider uses multiple DNS resolvers at the same time (up to three), depending on the user policy, and the way I am using this functionality is by reading and writing datagrams from intercepted flows within NRDNSProxyProvider. If let's say one out of three resolvers returned blocked subnets, then I return nxdomain in datagrams for this flow.
The problem here is that WebKit generated flows are handled in Content Filter before DNSProxy, which means that if I allow it by default, the flow is supposedly will go through, but because DNSProxy returns nxdomain for the flow, it freezes. However when I do the same thing with the socket flow, everything works well because socket flow handled in Content Filter after DNSProxy returned datagrams for the same flow.
Let's say I have google.com blocked. In this case if I will use some third party app like ICS Dig or Network Tools to send a request for specified domain, I will receive response status nxdomain, but I will try to do the same in Safari, the flow will freeze instead because Content Filter allowed it before DNSProxy blocked it
I use NEDNSProxyProvider for resolving domains using specified DNS resolvers. When I first started the project, I tried to implement the same functionality just using Content Filter Provider, but that didn’t work because NEFilterDataProvider didn’t support async operations
upd: I tried to defer the response for webkit generated flows in NEFilterDataProvider but that didn't work well because handleNewFlow method would not support async operations.
Additionally I tried to add a DNS lookup for flows that are not registered (first connection, when the DNS Proxy hasn't yet made a decision whether the domain is blocked or not), but I assume it didn't work for the same reason. Just ignoring webkit generated flows is no help because they're are just set to NEFilterNewFlowVerdict.allow() by default then.
question: Is it even possible to achieve this goal? And would it work if I form custom response packets for blocked domains (let's say add remediation page ip address) in DNS Proxy target?
Thank you for your response!
upd: I implemented custom filtering using datagrams for each flow (read datagrams for each flow and create custom packets for the response)
I see, thank you a lot. I tried to use user defaults with the init(suiteName:) a couple days ago as I realised it was the same problem as I filed earlier about rules storage access from different targets.
Thank you again!
Thank you! That helped a lot
@eskimo
I also tried to add a simple PassthroughSubject<Void, Never>() to just notify about changes in rules and subscription to log the events. If I add subscription in FilterUtilities class directly, it logs all the events, however the same subscription in Filter Control Provider is not logging anything.
final class FilterUtilities: NSObject, ObservableObject {
let rulePublisher = PassthroughSubject<Void, Never>()
private var subs = Set<AnyCancellable>()
// MARK: - Init
override init() {
super.init()
loadRules()
rulePublisher
.eraseToAnyPublisher()
.sink {
Logger.statistics.info("[FilterUtilities] - filterRules changed")
}
.store(in: &subs)
}
// some other code
}
class FilterControlProvider: NEFilterControlProvider {
// MARK: - Properties
let filterUtilities = FilterUtilities.shared
private var subs = Set<AnyCancellable>()
// MARK: - Init
override init() {
super.init()
filterUtilities.rulePublisher
.eraseToAnyPublisher()
.sink {
Logger.statistics.info("[FilterControlProvider] - filterRules changed")
}
.store(in: &subs)
}
// some other code
}
Not sure what's the problem here. Would be grateful if you could tell me what I'm doing wrong. Thank you
I first tried to use it like this. I am not sure what's the right usage of it
filterUtilities.$filterRules
.debounce(for: .seconds(0.5), scheduler: DispatchQueue.main)
.sink { [weak self] newValue in
self?.filterUtilities.loadRules()
self?.notifyRulesChanged()
}
.store(in: &subs)
Thank you for your response @eskimo, but it seems that I misjudged my previous implementation with just JSON file. I am trying to understand why it's not working properly before moving to CoreData usage.
I have a class that I use for managing filtering rules
final class FilterUtilities: NSObject {
// MARK: - Properties
/// Filter rules
@Published
var filterRules: [FilterRule] = [FilterRule.default]
/// Singleton property
static let shared = FilterUtilities()
// MARK: - Init
override init() {
super.init()
loadRules()
}
func getRule(_ socketFlow: NEFilterSocketFlow) -> FilterRule?
func addRule(rule: FilterRule)
func removeRule(domain: String)
func loadRules() {
if let sharedContainerURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: Const.appGroupIdentifier) {
let fileURL = sharedContainerURL.appendingPathComponent("\(Const.filterStorageKey).json")
do {
let data = try Data(contentsOf: fileURL)
let decoder = JSONDecoder()
filterRules = try decoder.decode([FilterRule].self, from: data)
} catch {
Logger.statistics.error("Error loading filter rules: \(error.localizedDescription, privacy: .public)")
}
}
}
/// Method to save rules
func saveRules() {
if let sharedContainerURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: Const.appGroupIdentifier) {
let fileURL = sharedContainerURL.appendingPathComponent("\(Const.filterStorageKey).json")
if let encodedRules = try? JSONEncoder().encode(filterRules) {
do {
try encodedRules.write(to: fileURL)
} catch {
Logger.statistics.error("[FilterUtilities] - Error saving filter rules: \(error.localizedDescription, privacy: .public)")
}
}
} else {
Logger.statistics.error("[FilterUtilities] - container access denied")
}
}
}
I noticed that when I call getRule from my Filter Data Provider it doesn't contain newly added rules even though logs in FilterUtilities show that rule is added to filterRules (func getRule basically returns the rule for matched domain), however, if I reload the app, the rules, that Filter Data Provider couldn't access before are now accessible. Then I tried to use Filter Control Provider in order to notify Filter Data provider that rules changed by first using notifyRulesChanged() function and when it didn't work, adding subscription on rules array like this, but that didn't work either. In my implementation rules are added from DNSProxy and then Filter Provider uses them to handle ip-connections. I understand that I am probably missing a very important part but I don't see what's causing this problem yet.
override init() {
super.init()
filterUtilities.$filterRules
.debounce(for: .seconds(0.5), scheduler: DispatchQueue.main)
.sink { [weak self] _ in
self?.filterUtilities.loadRules()
}
.store(in: &subs)
}
I thought the problem could be related to synchronisation of the rules adding/retrieving, but all the logs are showing that DNSProxy is adding rules properly, before the Filter Provider intercepts the flow.
I would be very grateful for any suggestions
Thank you!
How were you able to access the provider configurations in the container app using NEDNSProxyProviderProtocol().providerConfiguration?
I was able to access them using options array in startProxy but whenever I try to access from any other target using NEDNSProxyProviderProtocol().providerConfiguration, it returns nil
With a similar approach, I tried to write data to a JSON file in the app group container from my main target and read the file from the Filter Data Provider when needed (the Filter Control Provider observes changes), and it worked well. However, if I try to use Core Data, it still invalidates Content Filter. And there are no crash logs for any of the filter providers
I followed your advice and tried to first test access to the shared container and everything worked as expected: Filter Data and Filter Control Providers were able to read from app group container and main target was able to write data. After that I tried to add Core Data like this:
private lazy var persistentContainer: NSPersistentContainer = {
guard let sharedContainerURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "MyGroupIdentifier") else {
fatalError("Shared container is not accessible.")
}
let storeURL = URL.storeURL(group: "MyGroupIdentifier", database: "MyCoreDataModel")
let description = NSPersistentStoreDescription(url: storeURL)
let container = NSPersistentContainer(name: "MyCoreDataModel")
container.persistentStoreDescriptions = [description]
container.loadPersistentStores(completionHandler: { (storeDescription, error) in
if let error = error as NSError? {
fatalError("Unresolved error \(error), \(error.userInfo)")
}
})
return container
}()
public extension URL {
static func storeURL(group: String, database: String) -> URL {
guard let fileContainer = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: group) else {
fatalError("Shared file container could not be created.")
}
return fileContainer.appendingPathComponent("\(database).sqlite")
}
}
My VM in the main app target stores Model shared instance and everything compiled as expected. However, if I try to call let's say fetch for database, Content Filter becomes Invalid. I had similar problem, when I tried to add async operation to my Filter Data Provider handleNewFlow func in order to store intercepted flows, that's why I moved logic to VM and UserDefaults at that time