iOS Development

Taking part in AAC stream in AVAudioEngine

Spread the love


The problem is scheduling an AAC buffer in a AVAudioPlayerNode which solely consumes uncompressed linear pcm buffers.

The operate playAudio takes a Knowledge() object that incorporates a piece of AAC being streamed to the consumer, usually a number of kilobytes in dimension. The method goes like this:

  1. Write AAC chunk to a AVAudioCompressedBuffer.
  2. Create a pcm buffer and convert the AVAudioCompressedBuffer into it.
  3. Schedule the buffer within the Participant Node

That is my try. Nevertheless it doesn’t work, the audio simply doesn’t play in any respect within the audio system. I wrote a bunch of debugging statements throughout the capabilities to determine what’s going on.

    func preparePlayer() {
        guard let engine = audioEngine else {
            print("Audio engine shouldn't be initialized")
            return
        }

        // Initialize a participant node and connect it to the engine
        playerNode = AVAudioPlayerNode()

        engine.connect(playerNode!)
        print("Output Node Format: (engine.outputNode.outputFormat(forBus: 0))")

        pcmFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: 48000, channels: 1, interleaved: false)
        
        engine.join(playerNode!, to: engine.outputNode, format: pcmFormat)

        var asbd = AudioStreamBasicDescription()
        asbd.mSampleRate = 24000  // 24 kHz
        asbd.mFormatID = kAudioFormatMPEG4AAC // AAC
        asbd.mChannelsPerFrame = 1  // Mono
        asbd.mBytesPerPacket = 0  // Varies (compressed format)
        asbd.mFramesPerPacket = 1024 
        asbd.mBytesPerFrame = 0  // Varies (compressed format)
        asbd.mBitsPerChannel = 0  // Varies (compressed format)

        // Create an AVAudioFormat with the AudioStreamBasicDescription
        sourceFormat = AVAudioFormat(streamDescription: &asbd)
        
        // Initialize the audio converter with the supply (AAC) and vacation spot (PCM) codecs
        audioConverter = AVAudioConverter(from: sourceFormat!, to: pcmFormat!)
    }
    
    func playAudio(audioData: Knowledge) {
        guard let playerNode = self.playerNode else {
            print("Participant node shouldn't be initialized")
            return
        }
        guard let engine = audioEngine else {
            print("Audio engine shouldn't be initialized")
            return
        }
        
        let compressedBuffer = AVAudioCompressedBuffer(format: sourceFormat!, packetCapacity: 1024, maximumPacketSize: audioConverter!.maximumOutputPacketSize)
        compressedBuffer.byteLength = AVAudioPacketCount(audioData.depend)
        print("audioData incorporates (audioData.depend) counts")
        let middleIndex = audioData.depend / 2
        let middleRangeAudioData = middleIndex..<(middleIndex + 10)
        print("Center bytes of audioData: (Array(audioData[middleRangeAudioData]))")

        
        audioData.withUnsafeBytes {
            compressedBuffer.knowledge.copyMemory(from: $0.baseAddress!, byteCount: audioData.depend)
        }
        print("compressedBuffer incorporates (compressedBuffer.packetCount) packet counts")
        print("compressedBuffer incorporates (compressedBuffer.byteCapacity) byte capability")
        print("compressedBuffer incorporates (compressedBuffer.byteLength) legitimate bytes")
        print("compressedBuffer incorporates (compressedBuffer.packetCapacity) packet capability")
        let bufferPointer = compressedBuffer.knowledge.bindMemory(to: UInt8.self, capability: audioData.depend)
        let bufferBytes = Array(UnsafeBufferPointer(begin: bufferPointer, depend: audioData.depend))
        let middleRangeCompressedBuffer = middleIndex..<(middleIndex + 10)
        print("Center bytes of compressedBuffer: (Array(bufferBytes[middleRangeCompressedBuffer]))")


        // Create a PCM buffer
        let pcmBuffer = AVAudioPCMBuffer(pcmFormat: pcmFormat!, frameCapacity: 1024)

        let inputBlock: AVAudioConverterInputBlock = { inNumPackets, outStatus in
            outStatus.pointee = AVAudioConverterInputStatus.haveData
            return compressedBuffer
        }

        var error: NSError?
        let conversionResult = audioConverter!.convert(to: pcmBuffer!, error: &error, withInputFrom: inputBlock)
        if conversionResult == .error {
            print("Conversion failed with error: (String(describing: error))")
        } else {
            print("Conversion profitable")
            print("buffer incorporates (pcmBuffer?.frameLength ?? 123456) frames")
        }
                
        if let frameLength = pcmBuffer?.frameLength, frameLength > 0 {
            let channelCount = pcmFormat?.channelCount ?? 0
            for channel in 0..<channelCount {
                if let channelData = pcmBuffer?.floatChannelData?[Int(channel)] {
                    let channelDataPointer = UnsafeBufferPointer(begin: channelData, depend: Int(frameLength))
                    let firstFewSamples = Array(channelDataPointer.prefix(10))
                    print("First few samples of pcmBuffer in channel (channel): (firstFewSamples)")
                }
            }
        }

        
        if !engine.isRunning {
            do {
                strive engine.begin()
            } catch {
                print("Error beginning audio engine: (error)")
            }
        }

        playerNode.scheduleBuffer(pcmBuffer!, completionHandler: nil)
    }

That is an instance debugging log:

audioData incorporates 1369 counts
Center bytes of audioData: [250, 206, 86, 76, 254, 10, 221, 187, 190, 243]
compressedBuffer incorporates 0 packet counts
compressedBuffer incorporates 4096 byte capability
compressedBuffer incorporates 1369 legitimate bytes
compressedBuffer incorporates 1024 packet capability
Center bytes of compressedBuffer: [250, 206, 86, 76, 254, 10, 221, 187, 190, 243]
Conversion profitable
buffer incorporates 0 frames

The converter doesn’t output any errors, however the pcm buffer incorporates 0 frames. Wanting backwards that is most likely as a result of the compressedBuffer additionally does include 0 frames. The audio will get written to the compressedBuffer however there’s the place issues cease making sense. I’ve labored nearly all the time with pcm audio earlier than and since it’s linear I can interpolate what number of frames it has per packet primarily based on the audio settings, no huge deal. In compressed audio that’s not attainable as a result of bitrate is mostly variable. Perhaps this concern is expounded to having 0 frames, or possibly one thing else.

Leave a Reply

Your email address will not be published. Required fields are marked *