Post

Replies

Boosts

Views

Activity

Writing video using AVAssetWriter, AVAssetReader, and AVSPEECHSYNTHESIZER
Hello, First, some version and software details: Software: iOS 18.1 Hardware: iPhone 14 Pro Max and later Xcode: 16.0 Summary: AVAssetReader is not concatenating a video at the beginning of the output video. The output video should contain a scene of me introducing the content, followed by a blue screen with AVSpeechSynthesizer reading out a text that I pasted above the "Generate Video" button. Details: Now, let's talk about the app. Basically, I’m developing an app that generates a video with the following features: My app will create an output video that is split into an opening scene followed by a fully blue screen. The opening scene will be taken from a video I choose from my gallery. I will read the opening video using AVAssetReader as usual. After the opening scene, I will use the content of a text read by AVSpeechSynthesizer.write(). After the opening scene, the synthesized audio will start playing while the blue screen is displayed. All of this is already defined in the attached project. Each project file has a comment at the beginning introducing its content. How to test: Write something in the field above the "Generate Video" button. For example, type "Hello, world!" Then, press the "Library" button and select a video from the gallery, about 30 seconds long. That’s it. Press the "Generate Video" button. The result I’ve experienced is a crash or failure to generate the video. Practical example of what I want to achieve: Suppose I record a 30-second video where I say, "I’m going to tell you the story of Snow White." Then, I paste the "Snow White" story into the field above the "Generate Video" button. The output video should contain me saying, "I’m going to tell you the story of Snow White." After that, the AVSpeechSynthesizer will read the story I pasted, while displaying a blue screen. I look forward to a solution. Thank you very much! convertToCMSampleBuffer.swift convertToPixelBuffer.swift createInputs.swift createVideo.swift test.swift saveVideo.swift TestApp.swift editingVideo.swift sampleReaderProvider.swift misc.swift sampleProvider.swift
7
0
490
Oct ’24
CMSAMPLEBuffer: audio PCM to MP4 AAC
Hello, As explained in this link, the AVAssetReaderTrackOutput.copyNextSampleBuffer() returns a CMSampleBuffer in linear PCM audio format. I want to place this audio buffer into an AVAssetWriterInput of type kAudioFormatMPEG4AAC, but I can't manage the conversion. Could you help me by providing an extension that returns a CMSampleBuffer converted from linear PCM audio format to kAudioFormatMPEG4AAC? Example: extension CMSampleBuffer { func fromPCMToAAC() -> CMSampleBuffer? { // Here, get a new AudioStreamBasicDescription, create a CMSampleBuffer and a CMBlockBuffer } } I've tried multiple times but without success. Software: iOS 18.1 XCode: 16.0 Thank you!
1
0
252
Oct ’24
iOS: I need to convert CMSampleBuffer Linear PCM audio to MPEG-4 AAC
Hello, I am a deaf-blind wheelchair user, and I program in Swift using a braille display. I’m reaching out for your help on an issue I’ve been struggling to solve. Basically, when I extract a CMSampleBuffer from an AVAsset of a video, it comes with the Audio Format ID as Linear PCM. However, when I try to pass this CMSampleBuffer to write another video using AVAssetWriter, the video ends up muted. The audio settings of the output video are configured to MPEG-4 AAC, but the input CMSampleBuffer has the Audio Format ID as Linear PCM. I would like to request an extension for CMSampleBuffer that converts Linear PCM audio to MPEG-4 AAC. I’ve searched extensively and couldn’t find anything. Looking forward to your help. Thank you.
1
0
205
Sep ’24
Silent output MP4 when using AVAssetReaderHello, I am trying to create an MP4 by obtaining the content from another source MP4. The source MP4 would be read with `AVAssetReader` and the output written with `AVAssetWriter`. I wanted to do partial t
Hello, I am trying to create an MP4 by obtaining the content from another source MP4. The source MP4 would be read with AVAssetReader and the output written with AVAssetWriter. I wanted to do partial tests: first, I placed only the video in the output MP4. Now, I am trying to place only the audio in the output MP4. I even managed to get the output MP4 to have the same length (in seconds) as the source MP4. But the problem is simple: the output MP4 is simply silent. Naturally, I want it to have audio. Below are two excerpts from the source code. Reading and writing. Note: The variable videoURL is from the class where the function writeVideo() is located. Its assignment happens in another scope, already debugged. Snippet: let semaphore = DispatchSemaphore(value: 0) func writeVideo() { var audioReaderBuffers = [CMSampleBuffer]() // File directory url = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0].appendingPathComponent("*****/output.mp4") guard let url = url else { return } try FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true) if FileManager.default.fileExists(atPath: url.path()) { try FileManager.default.removeItem(at: url) } if let videoURL = videoURL { let videoAsset = AVAsset(url: videoURL) Task { let audioTrack = try await videoAsset.loadTracks(withMediaType: .audio).first! let reader = try AVAssetReader(asset: videoAsset) let audioSettings = [ AVFormatIDKey: kAudioFormatLinearPCM, AVSampleRateKey: 44100, AVNumberOfChannelsKey: 2 ] as [String : Any] let audioOutputTest = try await audioTrack.getAudioSettings() let readerAudioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: audioSettings) reader.add(readerAudioOutput) reader.startReading() while let sampleBuffer = readerAudioOutput.copyNextSampleBuffer() { audioReaderBuffers.append(sampleBuffer) } semaphore.signal() } } semaphore.wait() let audioInput = createAudioInput(sampleBuffer: audioReaderBuffers[0]) let assetWriter = try AVAssetWriter(outputURL: url, fileType: .mp4) assetWriter.add(audioInput) assetWriter.startWriting() assetWriter.startSession(atSourceTime: .zero) for (index, buffer) in audioReaderBuffers.enumerated() { while !audioInput.isReadyForMoreMediaData { usleep(1000) } audioInput.append(buffer) } assetWriter.finishWriting { switch assetWriter.status { case .completed: print("Operation completed successfully: \(url.absoluteString)") case .failed: if let error = assetWriter.error { print("Error description: \(error.localizedDescription)") } else { print("Error not found.") } default: print("Error not found.") } } } Below is the createAudioInput method: func createAudioInput(sampleBuffer: CMSampleBuffer) -> AVAssetWriterInput { let audioSettings = [ AVFormatIDKey: kAudioFormatMPEG4AAC, AVSampleRateKey: 48000, AVEncoderBitRatePerChannelKey: 64000, AVNumberOfChannelsKey: 1 ] as [String : Any] let audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioSettings, sourceFormatHint: sampleBuffer.formatDescription) audioInput.expectsMediaDataInRealTime = false return audioInput } I await your help, please.
0
0
250
Sep ’24
Add 30 frames per secons in assetWriter
Hello, I have converted UIImage to CVPixelBuffer. I am creating a video writing app. In some cases, the same CVPixelBuffer should last in the video for 2 seconds or more. However, I need to add 30 CVPixelBuffers per second because the video, to work on social media, must be 30 frames per second. The problem is that whenever I try to add frames to long videos, like 50-minute videos, it gives an error. The error is something like "Operation cannot be completed". Give me an example of a loop to add 30 CVPixelBuffers per second to a currently written video. Example: while true { if videoInput.isReadyForMoreMediaData { break } if videoInput.isReadyForMoreMediaData, let buffer = videoProvider.getNextFrame() { adaptor.append(buffer, withPresentationTime: CMTime(value: 1, timescale: 30)) } } I await your response.
0
0
481
Jun ’24
Accessibility: VoiceOver turns off by itself when I enter APPS recently compiled via XCODE
Hello, I'm experiencing a sad BUG. I am deaf, blind, in a wheelchair. I program in XCode via Braille display. I compile any APP in XCode, including the APP test "Hello world!" and I transfer this APP to my iPhone via USB cable or Wi-Fi. So, when I try to enter the APP, VoiceOver suddenly deactivates itself and no longer activates. I always have to turn off my iPhone and turn it on again. He only reacts with someone he sees as helping. I'm not programming in XCode, because I can't test APPS on my iPhone anymore. Help me. Software in Mac: XCode 15.4 beta Software MacOS: latest version; Software in iPhone:iOS 17.5 Hardware: iPhone 14 Pro Max The BUG happens always.
0
0
266
May ’24
I can't add more than 2 thousand and a few CMSAMPLEBUFFER in AVAssetWrite
Hello, I am deaf-blind and I program with a braille display. Currently, I am experiencing a difficulty with one of my APPs. Basically, I'm converting AVAudioPCMBuffer for CMSampleBuffer and so far so good. I want to add several CMSampleBuffer in a video written with AVAssetWrite . The problem is that I can only add up to more or less 2 thousands CMSampleBuffer . I'm trying to create a video. In this video, I put photos that are in an array and then I put audio from CMSampleBuffer. But I can't add many CMSampleBuffer and only goes up to 2 thousand something. I do not know what else to do. Help me. Below is a small excerpt of the code: let queue = DispatchQueue(label: "AssetWriterQueue") let audioProvider = SampleProvider(buffers: audioBuffers) let videoProvider = SampleProvider(buffers: videoBuffers) let audioInput = createAudioInput(audioBuffers: audioBuffers) let videoInput = createVideoInput(videoBuffers: videoBuffers) let adaptor = createPixelBufferAdaptor(videoInput: videoInput) let assetWriter = try AVAssetWriter(outputURL: url, fileType: .mp4) assetWriter.add(videoInput) assetWriter.add(audioInput) assetWriter.startWriting() assetWriter.startSession(atSourceTime: .zero) await withCheckedContinuation { continuation in videoInput.requestMediaDataWhenReady(on: queue) { let time = videoProvider.getPresentationTime() if let buffer = videoProvider.getNextBuffer() { adaptor.append(buffer, withPresentationTime: time) } else { videoInput.markAsFinished() continuation.resume() } } } await withCheckedContinuation { continuation in audioInput.requestMediaDataWhenReady(on: queue) { if let buffer = audioProvider.getNextBuffer() { audioInput.append(buffer) } else { audioInput.markAsFinished() continuation.resume() } } }
0
0
428
Dec ’23
How to Make Personal Voice Recording in My Languag
I've been deaf and blind for 15 years' I'm not good at pronunciation in English, since I don't hear what I say, much less hear it from others. When I went to read the phrases to record my personal voice in Accessibility > Personal Voice, the 150 phrases to read are in English' How do I record phrases in Brazilian Portuguese? I speak Portuguese well' My English is very bad in pronunciation and deafness contributed' Help me.
1
0
779
Aug ’23
Complete process of how to add my voice as speech synthesis
Hello, I am deaf and blind. So my Apple studies are in text vi aBraille. One question: how do I add my voice as voice synthesis? Do I have to record it somewhere first? What is the complete process, starting with recording my voice? Do I have to record my voice reading something and then add it as voice synthesis? What's the whole process of that? There is no text explaining this' I found one about authorizing personal voice, but not the whole process starting the recording and such' Thanks!
3
0
1.5k
Aug ’23
I can't add AVSPEECHSynthesisMarker in AVSpeechSynthesizer
Hello! I have been facing great difficulty for days now. Pretty frustrating for me. Simply, I can't add any array bookmarks when I use AVSpeechSynthesizer speech marker with AVSpeechSynthesisMarker but i'm not helping this in AVSpeechSynthesizer . Here is a simple code, I removed the utterance part: let synt = AVSpeechSynthesizer() synt.write(expression, toBufferCallback: {buffer in}, toMarkerCallback: {marks in mark.append(AVSpeechSynthesisMarker(bookmarkName: "Test1", atByteSampleOffset: 4)) } Note that this simple code what I want is to add a AVSpeechSynthesisMarker no AVSpeechSynthesizer . XCode 15 beta is saying that brands It is immutable for being the type 'to leave' . But then how am I going to add markers in AVSpeechSynthesizer ? Please help me. Thanks!
0
0
469
Aug ’23