Hello,
First, some version and software details:
Software: iOS 18.1
Hardware: iPhone 14 Pro Max and later
Xcode: 16.0
Summary: AVAssetReader is not concatenating a video at the beginning of the output video. The output video should contain a scene of me introducing the content, followed by a blue screen with AVSpeechSynthesizer reading out a text that I pasted above the "Generate Video" button.
Details:
Now, let's talk about the app.
Basically, I’m developing an app that generates a video with the following features:
My app will create an output video that is split into an opening scene followed by a fully blue screen.
The opening scene will be taken from a video I choose from my gallery.
I will read the opening video using AVAssetReader as usual.
After the opening scene, I will use the content of a text read by AVSpeechSynthesizer.write().
After the opening scene, the synthesized audio will start playing while the blue screen is displayed.
All of this is already defined in the attached project.
Each project file has a comment at the beginning introducing its content.
How to test:
Write something in the field above the "Generate Video" button. For example, type "Hello, world!"
Then, press the "Library" button and select a video from the gallery, about 30 seconds long.
That’s it. Press the "Generate Video" button.
The result I’ve experienced is a crash or failure to generate the video.
Practical example of what I want to achieve:
Suppose I record a 30-second video where I say, "I’m going to tell you the story of Snow White."
Then, I paste the "Snow White" story into the field above the "Generate Video" button.
The output video should contain me saying, "I’m going to tell you the story of Snow White."
After that, the AVSpeechSynthesizer will read the story I pasted, while displaying a blue screen.
I look forward to a solution.
Thank you very much!
convertToCMSampleBuffer.swift
convertToPixelBuffer.swift
createInputs.swift
createVideo.swift
test.swift
saveVideo.swift
TestApp.swift
editingVideo.swift
sampleReaderProvider.swift
misc.swift
sampleProvider.swift
Post
Replies
Boosts
Views
Activity
Hello,
As explained in this link, the AVAssetReaderTrackOutput.copyNextSampleBuffer() returns a CMSampleBuffer in linear PCM audio format.
I want to place this audio buffer into an AVAssetWriterInput of type kAudioFormatMPEG4AAC, but I can't manage the conversion.
Could you help me by providing an extension that returns a CMSampleBuffer converted from linear PCM audio format to kAudioFormatMPEG4AAC?
Example:
extension CMSampleBuffer {
func fromPCMToAAC() -> CMSampleBuffer? {
// Here, get a new AudioStreamBasicDescription, create a CMSampleBuffer and a CMBlockBuffer
}
}
I've tried multiple times but without success.
Software: iOS 18.1
XCode: 16.0
Thank you!
Hello,
I am a deaf-blind wheelchair user, and I program in Swift using a braille display.
I’m reaching out for your help on an issue I’ve been struggling to solve.
Basically, when I extract a CMSampleBuffer from an AVAsset of a video, it comes with the Audio Format ID as Linear PCM. However, when I try to pass this CMSampleBuffer to write another video using AVAssetWriter, the video ends up muted.
The audio settings of the output video are configured to MPEG-4 AAC, but the input CMSampleBuffer has the Audio Format ID as Linear PCM.
I would like to request an extension for CMSampleBuffer that converts Linear PCM audio to MPEG-4 AAC.
I’ve searched extensively and couldn’t find anything.
Looking forward to your help.
Thank you.
Hello,
I am trying to create an MP4 by obtaining the content from another source MP4.
The source MP4 would be read with AVAssetReader and the output written with AVAssetWriter.
I wanted to do partial tests: first, I placed only the video in the output MP4.
Now, I am trying to place only the audio in the output MP4.
I even managed to get the output MP4 to have the same length (in seconds) as the source MP4.
But the problem is simple: the output MP4 is simply silent.
Naturally, I want it to have audio.
Below are two excerpts from the source code.
Reading and writing.
Note: The variable videoURL is from the class where the function writeVideo() is located. Its assignment happens in another scope, already debugged.
Snippet:
let semaphore = DispatchSemaphore(value: 0)
func writeVideo() {
var audioReaderBuffers = [CMSampleBuffer]()
// File directory
url = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0].appendingPathComponent("*****/output.mp4")
guard let url = url else { return }
try FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true)
if FileManager.default.fileExists(atPath: url.path()) {
try FileManager.default.removeItem(at: url)
}
if let videoURL = videoURL {
let videoAsset = AVAsset(url: videoURL)
Task {
let audioTrack = try await videoAsset.loadTracks(withMediaType: .audio).first!
let reader = try AVAssetReader(asset: videoAsset)
let audioSettings = [
AVFormatIDKey: kAudioFormatLinearPCM,
AVSampleRateKey: 44100,
AVNumberOfChannelsKey: 2
] as [String : Any]
let audioOutputTest = try await audioTrack.getAudioSettings()
let readerAudioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: audioSettings)
reader.add(readerAudioOutput)
reader.startReading()
while let sampleBuffer = readerAudioOutput.copyNextSampleBuffer() {
audioReaderBuffers.append(sampleBuffer)
}
semaphore.signal()
}
}
semaphore.wait()
let audioInput = createAudioInput(sampleBuffer: audioReaderBuffers[0])
let assetWriter = try AVAssetWriter(outputURL: url, fileType: .mp4)
assetWriter.add(audioInput)
assetWriter.startWriting()
assetWriter.startSession(atSourceTime: .zero)
for (index, buffer) in audioReaderBuffers.enumerated() {
while !audioInput.isReadyForMoreMediaData {
usleep(1000)
}
audioInput.append(buffer)
}
assetWriter.finishWriting {
switch assetWriter.status {
case .completed:
print("Operation completed successfully: \(url.absoluteString)")
case .failed:
if let error = assetWriter.error {
print("Error description: \(error.localizedDescription)")
} else {
print("Error not found.")
}
default:
print("Error not found.")
}
}
}
Below is the createAudioInput method:
func createAudioInput(sampleBuffer: CMSampleBuffer) -> AVAssetWriterInput {
let audioSettings = [
AVFormatIDKey: kAudioFormatMPEG4AAC,
AVSampleRateKey: 48000,
AVEncoderBitRatePerChannelKey: 64000,
AVNumberOfChannelsKey: 1
] as [String : Any]
let audioInput = AVAssetWriterInput(mediaType: .audio, outputSettings: audioSettings, sourceFormatHint: sampleBuffer.formatDescription)
audioInput.expectsMediaDataInRealTime = false
return audioInput
}
I await your help, please.
Hello,
I have converted UIImage to CVPixelBuffer. I am creating a video writing app. In some cases, the same CVPixelBuffer should last in the video for 2 seconds or more.
However, I need to add 30 CVPixelBuffers per second because the video, to work on social media, must be 30 frames per second.
The problem is that whenever I try to add frames to long videos, like 50-minute videos, it gives an error.
The error is something like "Operation cannot be completed".
Give me an example of a loop to add 30 CVPixelBuffers per second to a currently written video.
Example:
while true {
if videoInput.isReadyForMoreMediaData {
break
}
if videoInput.isReadyForMoreMediaData,
let buffer = videoProvider.getNextFrame() {
adaptor.append(buffer, withPresentationTime: CMTime(value: 1, timescale: 30))
}
}
I await your response.
Hello, I'm experiencing a sad BUG. I am deaf, blind, in a wheelchair. I program in XCode via Braille display. I compile any APP in XCode, including the APP test "Hello world!" and I transfer this APP to my iPhone via USB cable or Wi-Fi. So, when I try to enter the APP, VoiceOver suddenly deactivates itself and no longer activates. I always have to turn off my iPhone and turn it on again. He only reacts with someone he sees as helping. I'm not programming in XCode, because I can't test APPS on my iPhone anymore. Help me.
Software in Mac: XCode 15.4 beta
Software MacOS: latest version;
Software in iPhone:iOS 17.5
Hardware: iPhone 14 Pro Max
The BUG happens always.
Hello, I'll be objective: when I compile any APP in my XCode and transfer the APP to my iPhone, including the test APP "Hello world!", whether via network or USB cable, when I open the APP it simply doesn't work and the iPhone crashes. Only that. My XCode is 15.3, iPhone 14 Pro Max, IOS 17.5, macOS latest version.
Hello! I'm trying to save videos asynchronously. I've already used performChanges without the completionHandler, but it didn't work. Can you give me an example? Consider that the variable with the file URL is named fileURL. What would this look like asynchronously?
Hello,
I am deaf-blind and I program with a braille display.
Currently, I am experiencing a difficulty with one of my APPs.
Basically, I'm converting
AVAudioPCMBuffer
for
CMSampleBuffer
and so far so good.
I want to add several
CMSampleBuffer
in a video
written with
AVAssetWrite
.
The problem is that I can only add up to more or less
2 thousands
CMSampleBuffer
.
I'm trying to create a video.
In this video, I put photos that are in an array and then I put audio from
CMSampleBuffer.
But I can't add many
CMSampleBuffer and only goes up to
2 thousand something.
I do not know what else to do.
Help me.
Below is a small excerpt of the code:
let queue = DispatchQueue(label: "AssetWriterQueue")
let audioProvider = SampleProvider(buffers: audioBuffers)
let videoProvider = SampleProvider(buffers: videoBuffers)
let audioInput = createAudioInput(audioBuffers: audioBuffers)
let videoInput = createVideoInput(videoBuffers: videoBuffers)
let adaptor = createPixelBufferAdaptor(videoInput: videoInput)
let assetWriter = try AVAssetWriter(outputURL: url, fileType: .mp4)
assetWriter.add(videoInput)
assetWriter.add(audioInput)
assetWriter.startWriting()
assetWriter.startSession(atSourceTime: .zero)
await withCheckedContinuation { continuation in
videoInput.requestMediaDataWhenReady(on: queue) {
let time = videoProvider.getPresentationTime()
if let buffer = videoProvider.getNextBuffer() {
adaptor.append(buffer, withPresentationTime: time)
} else {
videoInput.markAsFinished()
continuation.resume()
}
}
}
await withCheckedContinuation { continuation in
audioInput.requestMediaDataWhenReady(on: queue) {
if let buffer = audioProvider.getNextBuffer() {
audioInput.append(buffer)
} else {
audioInput.markAsFinished()
continuation.resume()
}
}
}
I've been deaf and blind for 15 years'
I'm not good at pronunciation in English, since I don't hear what I say, much less hear it from others.
When I went to read the phrases to record my personal voice in Accessibility > Personal
Voice, the 150 phrases to read are in English'
How do I record phrases in Brazilian Portuguese?
I speak Portuguese well'
My English is very bad in pronunciation and deafness contributed'
Help me.
Hello, I am deaf and blind. So my Apple studies are in text vi aBraille. One question: how do I add my voice as voice synthesis? Do I have to record it somewhere first? What is the complete process, starting with recording my voice?
Do I have to record my voice reading something and then add it as voice synthesis?
What's the whole process of that? There is no text explaining this'
I found one about authorizing personal voice, but not the whole process starting the recording and such'
Thanks!
Hello!
I have been facing great difficulty for days now.
Pretty frustrating for me.
Simply, I can't add any array
bookmarks
when I use
AVSpeechSynthesizer
speech marker with
AVSpeechSynthesisMarker
but i'm not helping this in
AVSpeechSynthesizer
.
Here is a simple code, I removed the utterance part:
let synt = AVSpeechSynthesizer()
synt.write(expression, toBufferCallback: {buffer in}, toMarkerCallback: {marks in
mark.append(AVSpeechSynthesisMarker(bookmarkName: "Test1", atByteSampleOffset: 4))
}
Note that this simple code what I want is to add a
AVSpeechSynthesisMarker
no
AVSpeechSynthesizer
.
XCode 15 beta is saying that
brands
It is
immutable
for being the type
'to leave'
.
But then how am I going to add markers in
AVSpeechSynthesizer
?
Please help me.
Thanks!