Wsrtjtyk

I am mapping DMA coherent memory from kernel to user space. At user level I use mmap() and in kernel driver I use dma_alloc_coherent() and afterwards remap_pfn_range() to remap the pages. This basically works as I can write data to the mapped area in my app and verify it in my kernel driver.

However, despite using dma_alloc_coherent (which should alloc uncached memory) and pgprot_noncached() the kernel informs me with this dmesg output:

map pfn ram range req uncached-minus for [mem 0xABC-0xCBA], got write-back

In my understanding, write-back is cached memory. But I need uncached memory for the DMA operation.

The Code (only showing the important parts):

User App

fd = open(dev_fn, O_RDWR | O_SYNC);

if (fd > 0)

{

    mem = mmap ( NULL 

               , mmap_len

               , PROT_READ | PROT_WRITE

               , MAP_SHARED

               , fd

               , 0

               );

}

For testing purposes I used mmap_len = getpagesize(); Which is 4096.

Kernel Driver

typedef struct

{

    size_t     mem_size;

    dma_addr_t dma_addr;

    void       *cpu_addr;

} Dma_Priv;



fops_mmap()

{

    dma_priv->mem_size = vma->vm_end - vma->vm_start;



    dma_priv->cpu_addr = dma_alloc_coherent ( &gen_dev

                                            , dma_priv->mem_size

                                            , &dma_priv->dma_addr

                                            , GFP_KERNEL

                                            );

    if (dma_priv->cpu_addr != NULL)

    {

        vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);

        remap_pfn_range ( vma

                        , vma->vm_start

                        , virt_to_phys(dma_priv->cpu_addr)>>PAGE_SHIFT

                        , dma_priv->mem_size

                        , vma->vm_page_prot

                        )

    }

}

Useful information I've found

1) PATting Linux: https://www.kernel.org/doc/ols/2008/ols2008v2-pages-135-144.pdf

Page 7 --> mmap with O_SYNC (uncached):

Applications can open /dev/mem with the O_SYNC flag and then do mmap
on it. With that, applications will be accessing that address with an
uncached memory type. mmap will succeed only if there is no other
conflicting mappings to the same region.

I used the flag, doesn't help.

Page 7 --> mmap without O_SYNC (uncached-minus):

mmap without O_SYNC, no existing mapping, and not a write-back region:
For an mmap that comes under this category, we use uncached-minus type
mapping. In the absence of any MTRR for this region, the effective
type will be uncached. But in cases where there is an MTRR, making
this region write-combine, then the effective type will be
write-combine.

2) pgprot_noncached()

In /arch/x86/include/asm/pgtable.h I found this:

#define pgprot_noncached(prot)                              

    ((boot_cpu_data.x86 > 3)                                

     ? (__pgprot(pgprot_val(prot) |                         

             cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) 

     : (prot))

Is it possible that x86 always sets a noncached request to UC_MINUS, which results in combination with MTRR in a cached write-back?

I am using Ubuntu 16.04.1, Kernel: 4.10.0-40-generic.

EDIT: SOLVED

https://www.kernel.org/doc/Documentation/x86/pat.txt

Drivers wanting to export some pages to userspace do it by using mmap
interface and a combination of 1) pgprot_noncached() 2)
io_remap_pfn_range() or remap_pfn_range() or vmf_insert_pfn()

With PAT support, a new API pgprot_writecombine is being added. So,
drivers can continue to use the above sequence, with either
pgprot_noncached() or pgprot_writecombine() in step 1, followed by
step 2.

In addition, step 2 internally tracks the region as UC or WC in
memtype list in order to ensure no conflicting mapping.

Note that this set of APIs only works with IO (non RAM) regions. If
driver wants to export a RAM region, it has to do set_memory_uc() or
set_memory_wc() as step 0 above and also track the usage of those
pages and use set_memory_wb() before the page is freed to free pool.

I added set_memory_uc() before pgprot_noncached() and it did the thing.

if (dma_priv->cpu_addr != NULL)

{

    set_memory_uc(dma_priv->cpu_addr, (dma_priv->mem_size/PAGE_SIZE));

    vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);

    remap_pfn_range ( vma

                    , vma->vm_start

                    , virt_to_phys(dma_priv->cpu_addr)>>PAGE_SHIFT

                    , dma_priv->mem_size

                    , vma->vm_page_prot

                    )

}