C++ Weekly - Ep 374 - C++23's out_ptr and inout_ptr

Подписаться 113 тыс.

Просмотров 18 тыс.

50% 1

☟☟ Awesome T-Shirts! Sponsors! Books! ☟☟
Upcoming Workshop: Applied constexpr: The Power of Compile-Time Resources, C++ Under The Sea, October 10, 2024
► cppunderthesea...
Discussion: github.com/lef...
T-SHIRTS AVAILABLE!
► The best C++ T-Shirts anywhere! my-store-d16a2...
WANT MORE JASON?
► My Training Classes: emptycrate.com/...
► Follow me on twitter: / lefticus
SUPPORT THE CHANNEL
► Patreon: / lefticus
► Github Sponsors: github.com/spo...
► Paypal Donation: www.paypal.com...
GET INVOLVED
► Video Idea List: github.com/lef...
JASON'S BOOKS
► C++23 Best Practices
Amazon Paperback: amzn.to/47MEAhj
Leanpub Ebook: leanpub.com/cp...
► C++ Best Practices
Amazon Paperback: amzn.to/3wpAU3Z
Leanpub Ebook: leanpub.com/cp...
JASON'S PUZZLE BOOKS
► Object Lifetime Puzzlers Book 1
Amazon Paperback: amzn.to/3g6Ervj
Leanpub Ebook: leanpub.com/ob...
► Object Lifetime Puzzlers Book 2
Amazon Paperback: amzn.to/3whdUDU
Leanpub Ebook: leanpub.com/ob...
► Object Lifetime Puzzlers Book 3
Leanpub Ebook: leanpub.com/ob...
► Copy and Reference Puzzlers Book 1
Amazon Paperback: amzn.to/3g7ZVb9
Leanpub Ebook: leanpub.com/co...
► Copy and Reference Puzzlers Book 2
Amazon Paperback: amzn.to/3X1LOIx
Leanpub Ebook: leanpub.com/co...
► Copy and Reference Puzzlers Book 3
Leanpub Ebook: leanpub.com/co...
► OpCode Puzzlers Book 1
Amazon Paperback: amzn.to/3KCNJg6
Leanpub Ebook: leanpub.com/op...
RECOMMENDED BOOKS
► Bjarne Stroustrup's A Tour of C++ (now with C++20/23!): amzn.to/3X4Wypr
AWESOME PROJECTS
► The C++ Starter Project - Gets you started with Best Practices Quickly - github.com/cpp...
► C++ Best Practices Forkable Coding Standards - github.com/cpp...
O'Reilly VIDEOS
► Inheritance and Polymorphism in C++ - www.oreilly.co...
► Learning C++ Best Practices - www.oreilly.co...

Опубликовано:

14 окт 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 96

@skeleton_craftGaming Год назад

Malloc returns NULL if it cannot allocate the memory. Therefore it is theoretically possible that the address pointed to by that pointer is a null pointer

@skeleton_craftGaming Год назад

Also why can you not delete memory that was malloced?

@Raspredval1337 Год назад

@@skeleton_craftGaming it calls a destructor; destructors only exitst in c++, not in c. Plus, smart pointers are meant to hold AN instance of an object, not a block of instances: that's why you sholdn't really use a smart pointer wrapper over a heap allocated buffer. In theory, the whole block should be deleted, but that smells like a potential UB since the smart pointer calls delete instead of delete[ ].

@KingKarEl100 Год назад

@@Raspredval1337 the deleter issue has been resolved and the correct deleter is used by the smart ptrs type since c++ 17 (possibly since c++14)

@N.... Год назад

@@KingKarEl100 This is not true. If you are not using `std::make_unique` or `std::make_shared` then you must absolutely take the time to implement the proper custom deleter for the memory you are wrapping in the smart pointer. It is not possible to autodetect what the correct thing to do is. Keep in mind a pointer given from a C API might not have been allocated from the heap at all, for example it may be from an internal stack array and must be returned by a corresponding release function that marks it as no longer in use.

@oracleoftroy Год назад

@@skeleton_craftGaming _unique_ptr_ handles NULL just fine. And _unique_ptr_ can also delete memory created by _malloc_ or any other allocator just fine as long as you tell it to use _free_ or whatever other release function is appropriate. However, it uses _delete_ by default, so you have to explicitly tell it to use something else.

@AlfredoCorrea Год назад

1) a well crafted C API will have a corresponding free function, e.g. call it`free_data`, to work with `get_data`, so you don’t have to guess how to free it. This might also incidentally prevent the use of `make_unique`. 2) probably the difference between `out` and `in_out` can be appreciated more when the pointer is `shared_ptr`

@florianwinter4892 Год назад

Inout can be justified by C APIs that replace or reallocate pointers and use a single in/out argument for that. Whether such an interface is good or bad and whether there should rather be separate in and out parameters is arguable, but C APIs like that do exist, and sometimes we have to use them.

@zanfur Год назад

Reset does not implicitly call release; it destructs any original object. Release doesn't destruct the object. That's why it's used for inout: it's assumed that the C API has taken ownership of the pointer.

@bloodb0ne Год назад

Quite useful, but the crazy part is that you can set a breakpoint inside the decltype'd lambda :D never thought about that 👍

@jewulo Год назад

Lambdas are just classes with overloaded () so it should not be a surprise. I think?

@redcrafterlppa303 Год назад

I never thought that a block {} could exist between it's just to bizarre to the point that I see c++ as a dying ship adding every weird syntax just to stay relevant and becoming totally unusable and unreadable in the process. You really need a phd to fully understand even 1 line of the std lib. C++'s downfall is that it has too much obscure niche syntax and is adding more every 3 years and it's never removing syntax to stay backwards compatible.

@cppweekly Год назад

Understand the lambda and understand C++

@__hannibaal__ Год назад

When i swimming inside STD Oceans, I found thinGs never found in __official__ Documentation_____.

@florianwinter4892 Год назад

@@redcrafterlppa303It's been a while since I've seen a university from the inside, but my impression has always been that actual academics with actual PhDs hate C++. If this has changed in the last 2 decades, then maybe it is a good sign and not a bad sign for the future of the language.

@warzchoantark3327 Год назад

release() call releases the ownership, but doesn't delete the object. So, using inout_ptr in your code would lead to memory leak if you don't save raw pointer value from unique pointer and don't call delete explicitly somewhere later

@N.... Год назад

It is meant to be used when the C API takes the old pointer and frees it internally and then replaces it with the new pointer (or when it does not modify the pointer at all)

@AgrossMX 10 месяцев назад

the point of these if that if you standardize their usage then it would be easier to reason about the lifetime of raw pointers from C (or anywhere else for that matter) from the perspective of the compiler and any tooling built over it.

@driedurchin Год назад

"Let's just hope that's doing the right thing" is the most C++ thing I've ever heard.

@ABaumstumpf Год назад

The description of out_ptr and inout_ptr are written as cryptic as if being unreadable was the goal.

@tonydelamancha5513 Год назад

microsoft has something similar in wil::out_param. but it has an issue were you can't use the pointer in the same direct scope, like inside of an 'if' statement. does this have the same issue?

@LemonChieff Год назад

```c int* result = malloc(sizeof (int)); *result = 42; // dereferencing NULL on malloc failure ``` Here is a fix: ```c int* result = malloc(sizeof (int)); if (result) *result = 42; ```

@sneakystabbalot6735 Год назад

Thanks!

@cppweekly Год назад

Thanks!

@stephenhowe4107 Год назад

What Alfredo Correa said below. Suppose the C get_data function lives inside a DLL or shared library and is statically linked with the C library. Then the wrong free() is invoked in the lambda. The free() called is the statically linked version from the C++ main() and not the free() statically linked version from DLL or shared library. So Alfredo is right. There should be a free_data() and this should be in the same location as the get_data() in order for the correct free() to be invoked. uinique_ptr should not use a lambda.

@verylongtrain Год назад

The smart-pointer should use a lambda, which calls free_data() in that case.

@stephenhowe4107 Год назад

@@verylongtrain Good call

@florianwinter4892 Год назад

Does this work for non-pointer types, such as file descriptors or other handles of type int or HANDLE (which is a pointer, but thats's an implementation detail...)? For example, I have written a generic handle class with a unique_ptr-like interface that can manage ownership of any kind of "handle" a C API might return, and I have used it for all sorts of handle types used by the Windows API, OpenGL, libCURL, etc., and I can see std::out_ptr as the solution for a lot of ugly code for obtaining the raw handles from C APIs. Smart COM pointers are another (Windows-specific) use case.

@florianwinter4892 Год назад

(Of course, I can just try it myself. This was rather a comment on the issue you brought up of "having to know what allocation function was used". I think this was not so much designed to be used with unique_ptr and malloc, but rather with custom smart pointer-like objects that also call C APIs to free resources. But of course, it does work with std::unique_ptr with custom deleters for pointer types)

@Ariccio123 Год назад

As a fun fact, you can annotate HANDLE parameters with SAL and have the static analyzer do all your checking because it is indeed just a void**

@lostphysicist Год назад

Sorry, Jason. This isn't the right approach. Not because your code doesn't compile or causes undefined behavior directly as is, but because of an implicit rule that you should never free an object outside the place that allocated it. This is actually a well-known rule in the plugins world. Meaning: the correct way to do this is have your C code provide the deallocation function, and then make your unique_ptr's destructor call that function. The reason for that is that C API usually crosses compiler boundaries, which means that a compiler that compiled the C part doesn't have to be compatible with the one running the C++ part except through the C API, which means that a compiler that adds metadata to malloc or new that doesn't violate the standard will lead to the other compiler doing undefined behavior. See for example how the memory size in new[] calls are determined on delete[]. It's compiler dependent. So allocating in one compiler and freeing in the other is generally undefined behavior.

@Zettymaster Год назад

to put this into more real terms: instead of doing a free() on the allocated object - the implementation of said free wont necessarily be the same as the one used for actually destroying the object. say, if it comes from a memory pool, now you'd accidentally free part of the memory pool, instead of just marking it as unused. so the "better" code would look something akin to : std::unique_ptr ptr: im mainly thinking of stuff like the Vulkan API in the back of my head, with say a VkDevice, which you need to free with vkDestroyDevice

@lostphysicist Год назад

@@Zettymaster Precisely. Many libraries with C APIs have such interfaces.

@michaelkovaliov8877 Год назад

let's say that you have a c function void allocate_device(int** dev); and a free function as you suggested void dellocate_device(int* dev); // or maybe 2 pointers? you may use the deallocate_device as your free function inside the lambda, thus achieving the same thing basically, no?

@peterbonnema8913 Год назад

If I read the docs right, the reset function also uses the deleted to delete the original piece of memory. So I would've expected two invocations of your custom deleter function.

@Raspredval1337 Год назад

but isn't it a bad c practice to return a preallocated mem block instead of wanting the user to provide one? Seems like a smelly solution for an even smellier problem.

@platin2148 Год назад

I think you typically have functions being called with NULL first then it returns the size it needs and then you pass the right sized allocation.Or they allow you to give/get the size of the buffer directly

@claytorpedo Год назад

What is missing from this example is that most C APIs that allocate like this also provide a paired deallocation function, so there is no need to know internal details of how the alloc/dealloc works. I haven't looked into out_ptr/inout_ptr and I'm curious why they were chosen over unique_resource, which seems more explicit about what it's supposed to do.

@TsvetanDimitrov1976 Год назад

most of the directx functions allocating buffers, textures, etc. work like that

@tikabass Год назад

It's a very commonly practiced bad practice.

@platin2148 Год назад

To return internally allocated memory yeah. But COM and also DBUS sort of do it but indirectly..

@phantom_stnd Год назад

Trying out a new c++ feature util you run into undefined behavior challenge

@ivanp_personal Год назад

malloc can return NULL - that's what you've missed in C function, you have to check malloc result before you use it.

@willwatts8103 Год назад

How is this palaver with out_ptr beneficial? Would it not be easier and clearer to declare a lambda that does the deallocation + transfer to the unique_ptr? I realise it is deliberately a piddly example, but the faff:meat ratio to use this seems unfavourable.

@N.... Год назад

It reduces the amount of code noise you have to contend with. Without std::out_ptr you currently have to write stuff like this which leaks a raw pointer into the enclosing scope: void* p{}; some_c_api_make(&p); std::unique_ptr ptr(p, some_c_api_unmake); Avoiding that variable leakage is nontrivial prior to C++23. With std::out_ptr we can avoid the extra variable: std::unique_ptr ptr(nullptr, some_c_api_unmake); some_c_api_make(std::out_ptr(ptr));

@dwarftoad Год назад

Once you create the unique_ptr and do whatever interaction with the C functions is needed, you can pass/store the unique_ptr the same as any other unique_ptr without having to remember when or how to deallocate the memory created in the C function, perhaps in a separate part of the code with no knowlege of the C API (no need to #include in for example). In the past we'd just have to include the C header and make sure to use the correct C API call in the destructor of any object storing it to deallocate the resources, plus checks if it is valid/set or ownership has been transferred to or from another object, and don't forget to null the pointer when neccesary, etc. etc. Or write C++ wrapper code that does this. Not a big deal but it is useful, if you are otherwise using unique_ptrs a lot.

@j_ondrusek Год назад

Funny how you used the endl and then corrected it after a cut 😂

@cppweekly Год назад

:) took 20 weeks for someone to notice that. Yeah, I was very confused during the recording of that video. It took a couple of reedits to get it together and at one point I had no idea why I wasn't getting output!

@Wizatek Год назад

All i hear is that you need to use these things if you're doing sketchy stuff

@platin2148 Год назад

They are now going full crazy aren’t they? When relative pointers based on more horrifying templates?

@oracleoftroy Год назад

No, this is a great addition, and it makes working with C apis in a safe way a lot easier.

@dwarftoad Год назад

Nobody's forcing you to use this feature.

@eudorian111 Год назад

But why would you not just call free() after calling get_data on a pointer?

@N.... Год назад

Because that's prone to mistakes and isn't exception-safe.

@eudorian111 Год назад

@@N.... But won't this example will ultimately call free as well? How does it provide more exception safety?

@oracleoftroy Год назад

@@eudorian111 // the C way SomeCStruct * p; auto error = allocate(&p); // use p; release(p); // Try to use the C api safely in C++ before C++23 SomeCStruct * temp_p; auto error = allocate(&temp_p); std::unique_ptr p(temp_p); // use p; // C++23 std::unique_ptr p; auto error = allocate(std::out_ptr(p)); // use p; Plus I have a utility that looks like: template struct deleter_from_fn { template constexpr void operator()(T *arg) const noexcept { fn(arg); } }; template using custom_unique_ptr = std::unique_ptr; This lets me write: custom_unique_ptr p; auto error = allocate(std::out_ptr(p)); // use p; That way it is really easy to add unique_ptrs to C objects and not have awkward layers trying to wrangle the two API's together.

@vorrnth8734 Год назад

get_data does not need to use malloc. It can use a different allocator.

@N.... Год назад

@@eudorian111 You wouldn't free/release the data until you were done with it. The process of using the data before then may result in an exception being thrown.

@_RMSG_ Год назад

It would be nice if we could interface with C safely by not interfacing with C 😅

@User-cv4ee Год назад

Is pointer to pointer a common thing in C? Most function I have seen return a handle/pointer.

@X_Baron Год назад

I think that the idiom used here is that the C function is supposed to return an error code, and that's why it has to take a pointer to a pointer as an argument (to "return" a pointer also).

@N.... Год назад

Yes, it is very common, especially in graphics programming where the graphics API uses your allocator internally and then gives you back pointers from within the structures it has allocated.

@AlfredoCorrea Год назад

Yes, it is common. anything that creates a new object in which you want to abstract the heap and/or abstract the bunching of object building and object allocation together uses this idiom. a (niche) example I have handy is the whole of the MPI (Message Passing Interfcae) API.

@sirhenrystalwart8303 Год назад

You see a similar thing in c++ with a reference to a smart pointer. Both are providing two levels of indirection which is needed to do something like reseating the pointer.

@Ariccio123 Год назад

SAL has entire groups of annotations just for handling double pointers for exactly this reason!

@Runoratsu Год назад

13:37 long. Nice.

@TsvetanDimitrov1976 Год назад

Using unique_ptr looks like a code smell to me, since the only reason to do it is to abuse the dtor with a custom deleter. It's probably just a biased opinion, but to me it's a lot clearer to read something like int* p = nullptr; ctor_func(&p);..... dtor_func(&p). I might wrap the whole thing into a separate class that calls the c api functions in the ctor and dtor, but that's about it.

@N.... Год назад

std::unique_ptr automates the process of "I might wrap the whole thing into a separate class that calls the c api functions in the ctor and dtor" and is much less prone to mistakes, all you have to do is provide a proper custom deleter and everything is handled for you with a consistent API and exception safety.

@TsvetanDimitrov1976 Год назад

@@N.... It automates calling the custom deleter yes, but doesn't automate the construction. Also, those c api functions are usually a whole family not just create/delete, so it makes perfect sense(to me) to wrap them in a class.

@N.... Год назад

@@TsvetanDimitrov1976 Fair, but even with a class it may still make sense to use `std::unique_ptr` as the class member.

@dwarftoad Год назад

unique_ptr sometimes lets you avoid having to write a destructor or maybe any constructors at all, which can be nice.

@TsvetanDimitrov1976 Год назад

@@dwarftoad sure, but let's assume i want to create a dx12 texture. I'll need a descriptor describing it which i probably want to keep as invariant, also need to create the resource views this texture will be used with, and probably some additional things. I surely can keep all of this as separate objects, but they are intrinsically coupled with the texture i'm creating. Why should I keep them separate instead of creating a class that keeps all those things and their lifetimes coupled with the texture itself instead of writing a wrapper, with a proper ctor and dtor? Keep in mind that the wholoe "c api" boundary is most of the time caused by c++ breaking abi all the time, it's c++ internally in the library.

@lukasz2345 Год назад

One more reason to learn rust I guess :/

@KingKarEl100 Год назад

This feels like an unnecessary abstraction offered by the language. Since C++ 11, smart ptrs already offer you the ability to safely manage the lifetime of ptrs from a C library. The extra ptr types add more complexity to a problem thats already solved

@WenirR Год назад

It's not an additional pointer, it's a helper /thing/ to reduce boilerplate

@KingKarEl100 Год назад

@@WenirR actually they did introduce additional PTR wrapper types for this. The video shows one of them called "out_ptr_t". I understand it's used to reduce boiler plate but I don't think this is beneficial to the language. This same thing can be done by declaring a raw PTR, passing it to the c function then giving the raw ptr to the smart PTR with a custom deleter to manage the lifetime. We're doing almost the exact same simple steps. Maybe it's the example used in the video but this feels like a pointless addition

@WenirR Год назад

@@KingKarEl100 that's why I wrote /thing/ - I don't really care how they achieve it

@sergeiborodin9254 Год назад

That looks like a horrible crutch that will makes code even less comprehensible. Can't see any reason to implement such thing in a first place. C functions have so much nuances so almost every time you have explicitly create some kind of wrapper around C functions with memory allocations for C++ . Explicit custom wrapper class is 100 times more readable and usable. You can add things like validity checks, mutex sync and errno wrappers for try-catch. I already imagine this thing with Linux API. My rule of thumb: if you have to deal with raw pointers and C-allocations in C++ - you have to deal with allocation errors, store boundaries and write converters to C++ types.

@gregdee9085 Год назад

Ppl adding crap to C++ now because they're looking for something todo... C++ was mature ages ago, no amount of new features will help a programmer with bad judgement....

@MarekKnapek Год назад

Now the joy of `get_data` using `malloc` from one instance of CRT and your app using `free` from another version of CRT. Haha, Fun Will Now Commence. Let me explain: If you are interacting with C-style API, it is probably implemented inside other DLL. Windows has no concept of libc, C run-time library (as it is known on Windows or CRT for short) is just another library, just like any other library, Windows has no special treatment for libc. So, programs could either statically link against CRT (each EXE and each DLL could link against different version). Or dynamically link against CRT (it is also known as VCRedist) also each EXE and each DLL against different version. Or each application could distribute its own local copy of dynamically linkable CRT. It is a mess. Windows' dynamic linker also works a little bit differently than Linux's. So, to be at the safe side you need to: Either call `get_data` and `free_data` using the same DLL, do not `get_data` using DLL and `free` in your EXE. This works in all cases (static CRT, dynamic CRT, different versions of dynamic CRT). Or, if you are author of both EXE and DLL, set both projects to use dynamic CRT. Then you are fine to `malloc` in one and `free` in other, as both modules are sharing single CRT state (think of set of global variables) in separate CRT DLL. Also it is a good idea in general to link against dynamic CRT as it could be updated by Windows Update independently of you, fixing bugs and improving correctness and performance for free.

@vorrnth8734 Год назад

Usually the get_data also comes paired with a free_data. Use that like you are supposed to.

@dwarftoad Год назад

If the library provides a free_data() function, then you can just give that to unique_ptr e.g. std::unique_ptr p(nullptr, &free_data); (Or call it from the lambda instead of free() if you want to do it that way.)

@MarekKnapek Год назад

@@dwarftoad Yes, that was my point. If you use `get_data` from a library, you should also use `free_data` from the same library. Unless documented otherwise.

@MarekKnapek Год назад

@@vorrnth8734 Yes, that was my point. If you use `get_data` from a library, you should also use `free_data` from the same library. Unless documented otherwise.