I have the following code.
#include <iostream>
int * foo()
{
int a = 5;
return &a;
}
int main()
{
int* p = foo();
std::cout << *p;
*p = 8;
std::cout << *p;
}
And the code is just running with no runtime exceptions!
The output was 58
How can it be? Isn't the memory of a local variable inaccessible outside its function?
How can it be? Isn't the memory of a local variable inaccessible outside its function?
You rent a hotel room. You put a book in the top drawer of the bedside table and go to sleep. You check out the next morning, but "forget" to give back your key. You steal the key!
A week later, you return to the hotel, do not check in, sneak into your old room with your stolen key, and look in the drawer. Your book is still there. Astonishing!
How can that be? Aren't the contents of a hotel room drawer inaccessible if you haven't rented the room?
Well, obviously that scenario can happen in the real world no problem. There is no mysterious force that causes your book to disappear when you are no longer authorized to be in the room. Nor is there a mysterious force that prevents you from entering a room with a stolen key.
The hotel management is not required to remove your book. You didn't make a contract with them that said that if you leave stuff behind, they'll shred it for you. If you illegally re-enter your room with a stolen key to get it back, the hotel security staff is not required to catch you sneaking in. You didn't make a contract with them that said "if I try to sneak back into my room later, you are required to stop me." Rather, you signed a contract with them that said "I promise not to sneak back into my room later", a contract which you broke.
In this situation anything can happen. The book can be there—you got lucky. Someone else's book can be there and yours could be in the hotel's furnace. Someone could be there right when you come in, tearing your book to pieces. The hotel could have removed the table and book entirely and replaced it with a wardrobe. The entire hotel could be just about to be torn down and replaced with a football stadium, and you are going to die in an explosion while you are sneaking around.
You don't know what is going to happen; when you checked out of the hotel and stole a key to illegally use later, you gave up the right to live in a predictable, safe world because you chose to break the rules of the system.
C++ is not a safe language. It will cheerfully allow you to break the rules of the system. If you try to do something illegal and foolish like going back into a room you're not authorized to be in and rummaging through a desk that might not even be there anymore, C++ is not going to stop you. Safer languages than C++ solve this problem by restricting your power—by having much stricter control over keys, for example.
UPDATE
Holy goodness, this answer is getting a lot of attention. (I'm not sure why—I considered it to be just a "fun" little analogy, but whatever.)
I thought it might be germane to update this a bit with a few more technical thoughts.
Compilers are in the business of generating code which manages the storage of the data manipulated by that program. There are lots of different ways of generating code to manage memory, but over time two basic techniques have become entrenched.
The first is to have some sort of "long lived" storage area where the "lifetime" of each byte in the storage—that is, the period of time when it is validly associated with some program variable—cannot be easily predicted ahead of time. The compiler generates calls into a "heap manager" that knows how to dynamically allocate storage when it is needed and reclaim it when it is no longer needed.
The second method is to have a “short-lived” storage area where the lifetime of each byte is well known. Here, the lifetimes follow a “nesting” pattern. The longest-lived of these short-lived variables will be allocated before any other short-lived variables, and will be freed last. Shorter-lived variables will be allocated after the longest-lived ones, and will be freed before them. The lifetime of these shorter-lived variables is “nested” within the lifetime of longer-lived ones.
Local variables follow the latter pattern; when a method is entered, its local variables come alive. When that method calls another method, the new method's local variables come alive. They'll be dead before the first method's local variables are dead. The relative order of the beginnings and endings of lifetimes of storages associated with local variables can be worked out ahead of time.
For this reason, local variables are usually generated as storage on a "stack" data structure, because a stack has the property that the first thing pushed on it is going to be the last thing popped off.
It's like the hotel decides to only rent out rooms sequentially, and you can't check out until everyone with a room number higher than you has checked out.
So let's think about the stack. In many operating systems you get one stack per thread and the stack is allocated to be a certain fixed size. When you call a method, stuff is pushed onto the stack. If you then pass a pointer to the stack back out of your method, as the original poster does here, that's just a pointer to the middle of some entirely valid million-byte memory block. In our analogy, you check out of the hotel; when you do, you just checked out of the highest-numbered occupied room. If no one else checks in after you, and you go back to your room illegally, all your stuff is guaranteed to still be there in this particular hotel.
We use stacks for temporary stores because they are really cheap and easy. An implementation of C++ is not required to use a stack for storage of locals; it could use the heap. It doesn't, because that would make the program slower.
An implementation of C++ is not required to leave the garbage you left on the stack untouched so that you can come back for it later illegally; it is perfectly legal for the compiler to generate code that turns back to zero everything in the "room" that you just vacated. It doesn't because again, that would be expensive.
An implementation of C++ is not required to ensure that when the stack logically shrinks, the addresses that used to be valid are still mapped into memory. The implementation is allowed to tell the operating system "we're done using this page of stack now. Until I say otherwise, issue an exception that destroys the process if anyone touches the previously-valid stack page". Again, implementations do not actually do that because it is slow and unnecessary.
Instead, implementations let you make mistakes and get away with it. Most of the time. Until one day something truly awful goes wrong and the process explodes.
This is problematic. There are a lot of rules and it is very easy to break them accidentally. I certainly have many times. And worse, the problem often only surfaces when memory is detected to be corrupt billions of nanoseconds after the corruption happened, when it is very hard to figure out who messed it up.
More memory-safe languages solve this problem by restricting your power. In "normal" C# there simply is no way to take the address of a local and return it or store it for later. You can take the address of a local, but the language is cleverly designed so that it is impossible to use it after the lifetime of the local ends. In order to take the address of a local and pass it back, you have to put the compiler in a special "unsafe" mode, and put the word "unsafe" in your program, to call attention to the fact that you are probably doing something dangerous that could be breaking the rules.
For further reading:
What if C# did allow returning references? Coincidentally that is the subject of today's blog post:
Ref returns and ref locals
Why do we use stacks to manage memory? Are value types in C# always stored on the stack? How does virtual memory work? And many more topics in how the C# memory manager works. Many of these articles are also germane to C++ programmers:
Memory management
You're are simply reading and writing to memory that used to be the address of a. Now that you're outside of foo, it's just a pointer to some random memory area. It just so happens that in your example, that memory area does exist and nothing else is using it at the moment.
You don't break anything by continuing to use it, and nothing else has overwritten it yet. Therefore, the 5 is still there. In a real program, that memory would be reused almost immediately and you'd break something by doing this (though the symptoms may not appear until much later!).
When you return from foo, you tell the OS that you're no longer using that memory and it can be reassigned to something else. If you're lucky and it never does get reassigned, and the OS doesn't catch you using it again, then you'll get away with the lie. Chances are though you'll end up writing over whatever else ends up with that address.
Now if you're wondering why the compiler doesn't complain, it's probably because foo got eliminated by optimization. It usually will warn you about this sort of thing. C assumes you know what you're doing though, and technically you haven't violated scope here (there's no reference to a itself outside of foo), only memory access rules, which only triggers a warning rather than an error.
In short: this won't usually work, but sometimes will by chance.
Because the storage space wasn't stomped on just yet. Don't count on that behavior.
A little addition to all the answers:
If you do something like this:
#include <stdio.h>
#include <stdlib.h>
int * foo(){
int a = 5;
return &a;
}
void boo(){
int a = 7;
}
int main(){
int * p = foo();
boo();
printf("%d\n", *p);
}
The output probably will be: 7
That is because after returning from foo() the stack is freed and then reused by boo().
If you disassemble the executable, you will see it clearly.
In C++, you can access any address, but it doesn't mean you should. The address you are accessing is no longer valid. It works because nothing else scrambled the memory after foo returned, but it could crash under many circumstances. Try analyzing your program with Valgrind, or even just compiling it optimized, and see...
You never throw a C++ exception by accessing invalid memory. You are just giving an example of the general idea of referencing an arbitrary memory location. I could do the same like this:
unsigned int q = 123456;
*(double*)(q) = 1.2;
Here I am simply treating 123456 as the address of a double and write to it. Any number of things could happen:
q might in fact genuinely be a valid address of a double, e.g. double p; q = &p;.
q might point somewhere inside allocated memory and I just overwrite 8 bytes in there.
q points outside allocated memory and the operating system's memory manager sends a segmentation fault signal to my program, causing the runtime to terminate it.
You win the lottery.
The way you set it up it is a bit more reasonable that the returned address points into a valid area of memory, as it will probably just be a little further down the stack, but it is still an invalid location that you cannot access in a deterministic fashion.
Nobody will automatically check the semantic validity of memory addresses like that for you during normal program execution. However, a memory debugger such as Valgrind will happily do this, so you should run your program through it and witness the errors.
Did you compile your program with the optimiser enabled? The foo() function is quite simple and might have been inlined or replaced in the resulting code.
But I agree with Mark B that the resulting behavior is undefined.
Your problem has nothing to do with scope. In the code you show, the function main does not see the names in the function foo, so you can't access a in foo directly with this name outside foo.
The problem you are having is why the program doesn't signal an error when referencing illegal memory. This is because C++ standards does not specify a very clear boundary between illegal memory and legal memory. Referencing something in popped out stack sometimes causes error and sometimes not. It depends. Don't count on this behavior. Assume it will always result in error when you program, but assume it will never signal error when you debug.
Pay attention to all warnings. Do not only solve errors.
GCC shows this warning:
warning: address of local variable 'a' returned
This is the power of C++. You should care about memory. With the -Werror flag, this warning became an error and now you have to debug it.
It works because the stack has not been altered (yet) since a was put there.
Call a few other functions (which are also calling other functions) before accessing a again and you will probably not be so lucky anymore... ;-)
You are just returning a memory address. It's allowed, but it's probably an error.
Yes, if you try to dereference that memory address you will have undefined behavior.
int * ref () {
int tmp = 100;
return &tmp;
}
int main () {
int * a = ref();
// Up until this point there is defined results
// You can even print the address returned
// but yes probably a bug
cout << *a << endl;//Undefined results
}
This behavior is undefined, as Alex pointed out. In fact, most compilers will warn against doing this, because it's an easy way to get crashes.
For an example of the kind of spooky behavior you are likely to get, try this sample:
int *a()
{
int x = 5;
return &x;
}
void b( int *c )
{
int y = 29;
*c = 123;
cout << "y=" << y << endl;
}
int main()
{
b( a() );
return 0;
}
This prints out "y=123", but your results may vary (really!). Your pointer is clobbering other, unrelated local variables.
That's classic undefined behaviour that's been discussed here not two days ago -- search around the site for a bit. In a nutshell, you were lucky, but anything could have happened and your code is making invalid access to memory.
You actually invoked undefined behaviour.
Returning the address of a temporary works, but as temporaries are destroyed at the end of a function the results of accessing them will be undefined.
So you did not modify a but rather the memory location where a once was. This difference is very similar to the difference between crashing and not crashing.
In typical compiler implementations, you can think of the code as "print out the value of the memory block with adress that used to be occupied by a". Also, if you add a new function invocation to a function that constains a local int it's a good chance that the value of a (or the memory address that a used to point to) changes. This happens because the stack will be overwritten with a new frame containing different data.
However, this is undefined behaviour and you should not rely on it to work!
It can, because a is a variable allocated temporarily for the lifetime of its scope (foo function). After you return from foo the memory is free and can be overwritten.
What you're doing is described as undefined behavior. The result cannot be predicted.
The things with correct (?) console output can change dramatically if you use ::printf but not cout.
You can play around with debugger within below code (tested on x86, 32-bit, Visual Studio):
char* foo()
{
char buf[10];
::strcpy(buf, "TEST");
return buf;
}
int main()
{
char* s = foo(); // Place breakpoint and the check 's' variable here
::printf("%s\n", s);
}
It's the 'dirty' way of using memory addresses. When you return an address (pointer) you don't know whether it belongs to local scope of a function. It's just an address.
Now that you invoked the 'foo' function, that address (memory location) of 'a' was already allocated there in the (safely, for now at least) addressable memory of your application (process).
After the 'foo' function returned, the address of 'a' can be considered 'dirty', but it's there, not cleaned up, nor disturbed/modified by expressions in other part of program (in this specific case at least).
A C/C++ compiler doesn't stop you from such 'dirty' access (it might warn you though, if you care). You can safely use (update) any memory location that is in the data segment of your program instance (process) unless you protect the address by some means.
After returning from a function, all identifiers are destroyed instead of kept values in a memory location and we can not locate the values without having an identifier. But that location still contains the value stored by previous function.
So, here function foo() is returning the address of a and a is destroyed after returning its address. And you can access the modified value through that returned address.
Let me take a real world example:
Suppose a man hides money at a location and tells you the location. After some time, the man who had told you the money location dies. But still you have the access of that hidden money.
Your code is very risky. You are creating a local variable (which is considered destroyed after function ends) and you return the address of memory of that variable after it is destroyed.
That means the memory address could be valid or not, and your code will be vulnerable to possible memory address issues (for example, a segmentation fault).
This means that you are doing a very bad thing, because you are passing a memory address to a pointer which is not trustable at all.
Consider this example, instead, and test it:
int * foo()
{
int *x = new int;
*x = 5;
return x;
}
int main()
{
int* p = foo();
std::cout << *p << "\n"; // Better to put a newline in the output, IMO
*p = 8;
std::cout << *p;
delete p;
return 0;
}
Unlike your example, with this example you are:
allocating memory for an int into a local function
that memory address is still valid also when function expires (it is not deleted by anyone)
the memory address is trustable (that memory block is not considered free, so it will be not overridden until it is deleted)
the memory address should be deleted when not used. (see the delete at the end of the program)
I will start this by saying that on iOS this algorithm takes, on average, <2 seconds to complete and given a simpler, more specific input that is the same between how I test it on iOS vs. Android it takes 0.09 seconds and 2.5 seconds respectively, and the Android version simply quits on me, no idea if that would be significantly longer. (The test data gives the sorting algorithm a relatively simple task)
More specifically, I have a HashMap (Using an NSMutableDictionary on iOS) that maps a unique key(Its a string of only integers called its course. For example: "12345") used to get specific sections under a course title. The hash map knows what course a specific section falls under because each section has a value "Course". Once they are retrieved these section objects are compared, to see if they can fit into a schedule together based on user input and their "timeBegin", "timeEnd", and "days" values.
For Example: If I asked for schedules with only the Course ABC1234(There are 50 different time slots or "sections" under that course title) and DEF5678(50 sections) it will iterate through the Hashmap to find every section that falls under those two courses. Then it will sort them into schedules of two classes each(one ABC1234 and one DEF5678) If no two courses have a conflict then a total of 2500(50*50) schedules are possible.
These "schedules" (Stored in ArrayLists since the number of user inputs varies from 1-8 and possible number of results varies from 1-100,000. The group of all schedules is a double ArrayList that looks like this ArrayList>. On iOS I use NSMutableArray) are then fed into the intent that is the next Activity. This Activity (Fragment techincally?) will be a pager that allows the user to scroll through the different combinations.
I copied the method of search and sort exactly as it is in iOS(This may not be the right thing to do since the languages and data structures may be fundamentally different) and it works correctly with small output but when it gets too large it can't handle it.
So is multithreading the answer? Should I use something other than a HashMap? Something other than ArrayLists? I only assume multithreading because the errors indicate that too much is being done on the main thread. I've also read that there is a limit to the size of data passed using Intents but I have no idea.
If I was unclear on anything feel free to ask for clarification. Also, I've been doing Android for ~2 weeks so I may completely off track but hopefully not, this is a fully functional and complete app in the iTunes Store already so I don't think I'm that far off. Thanks!
1) I think you should go with AsynTask of Android .The way it handle the View into `UI
threadandBackground threadfor operations (Like Sorting` ) is sufficient enough to help
you to get the Data Processed into Background thread And on Processing you can get the
Content on UI Thread.
Follow This ShorHand Example for This:
Example to Use Asyntask
2) Example(How to Proceed):
a) define your view into onPreExecute()
b) Do your Background Operation into doInBackground()
c) Get the Result into onPostExceute() and call the content for New Activty
Hope this could help...
I think it's better for you to use TreeMap instead of HashMap, which sorts data automatically everytime you mutate it. Therefore you won't have to sort your data before start another activity, you just pass it and that's all.
Also for using it you have to implement Comparable interface in your class which represents value of Map.
You can also read about TreeMap class there:
http://docs.oracle.com/javase/7/docs/api/java/util/TreeMap.html
I am writing an OpenGL game and have a list of objects to render in the rendering loop, at the same time, updates from the server update this list of objects to render in a asynchronous task.
If I pause the rendering of the objects while the array gets updated obviously you see that on screen.
Would making a copy of the list, updating that and then copying it back (pause the render) be the best way?
Try the CopyOnWriteArrayList, which is a thread-safe version of ArrayList, which makes it possible to add elements to the list while traversing it.
A CopyOnWriteArrayList will allow multiple threads to access the list at once, as #Egor suggested, but I'm not sure it'll be fast enough.
Both reader and writer will interfere with each other all the time, and your users might notice it.
Give it a try. If it works - great, if not, you should have three copies of the list - one the reader (your rendering loop) accesses, one waiting for the next iteration of the rendering loop and another updated by the writer (the server update thread).
Use the three lists like so:
List<info> _readerList, _waitingList, _writerList;
In your rendering loop:
while(true) {
if(_waitingList!=_readerList)
_readerList = _waitingList
render list
}
In your service update thread:
while(true) {
read data from server
update _writerList
if there were updates {
_waitingList = _writerList
}
}
Before you start rendering, initialize _waitingList and _writerList to be two different lists with the same content, and start the loops.
This way you have no locking at all, and your two threads don't interfere with each other. The only point of contact between the two threads is the _waitingList reference, and both threads change that in one atomic operation.
The down side for this is that you'll have to wait until both the render loop and the server thread complete an iteration before the user sees the result.
CLARIFICATION:
It just occured to me that I missed an important point -the writer should create a new list and not reuse the same instance, otherwise after a couple of iterations both reader and writer will use the same list, and you're back to the same old race condition.
In a game for Android written in Scala, I have plenty of objects that I want to pool. First I tried to have both active (visible) and non active instances in the same pool; this was slow due to filtering that both causes GC and is slow.
So I moved to using two data structures, so when I need to get a free instance, I just take the first from the passive pool and add it to the active pool. I also fast random access to the active pool (when I need to hide an instance). I'm using two ArrayBuffers for this.
So my question is: which data structure would be best for this situation? And how should that (or those) specific data structure(s) be used to add and remove to avoid GC as much as possible and be efficient on Android (memory and cpu constraints)?
The best data structure is an internal list, where you add
var next: MyClass
to every class. The non-active instances then become what's typically called a "free list", while the active ones become a singly-linked list a la List.
This way your overhead is exactly one pointer per object (you can't really get any less than that), and there is no allocation or GC at all. (Unless you want to implement your own by throwing away part or all of the free list if it gets too long.)
You do lose some collections niceness, but you can just make your class be an iterator:
def hasNext = (next != null)
is all you need given that var. (Well, and extends Iterator[MyClass].) If your pool sizes are really quite small, sequential scanning will be fast enough.
If your active pool is too large for sequential scanning down a linked list and elements are not often added or deleted, then you should store them in an ArrayBuffer (which knows how to remove elements when needed). Once you remove an item, throw it on the free list.
If your active pool turns over rapidly (i.e. the number of adds/deletes is similar to the number of random accesses), then you need some sort of hierarchical structure. Scala provides an immutable one that works pretty well in Vector, but no mutable one (as of 2.9); Java also doesn't have something that's really suitable. If you wanted to build your own, a red-black or AVL tree with nodes that keep track of the number of left children is probably the way to go. (It's then a trivial matter to access by index.)
I guess I'll mention my idea. The filter and map methods iterate over the entire collection anyway, so you may as well simplify that and just do a naive scan over your collection (to look for active instances). See here: https://github.com/scala/scala/blob/v2.9.2/src/library/scala/collection/TraversableLike.scala
def filter(p: A => Boolean): Repr = {
val b = newBuilder
for (x <- this)
if (p(x)) b += x
b.result
}
I ran some tests, using a naive scan of n=31 (so I wouldn't have to keep more than a 32 bit Int bitmap), a filter/foreach scan, and a filter/map scan, and a bitmap scan, and randomly assigning 33% of the set to active. I had a running counter to double check that I wasn't cheating by not looking at the right values or something. By the way, this is not running on Android.
Depending on the number of active values, my loop took more time.
Results:
naive scanned a million times in: 197 ms (sanity check: 9000000)
filter/foreach scanned a million times in: 441 ms (sanity check: 9000000)
map scanned a million times in: 816 ms (sanity check: 9000000)
bitmap scanned a million times in: 351 ms (sanity check: 9000000)
Code here--feel free to rip it apart or tell me if there's a better way--I'm fairly new to scala so my feelings won't be hurt: https://github.com/wfreeman/ScalaScanPerformance/blob/master/src/main/scala/scanperformance/ScanPerformance.scala
I am not fully understanding what the synchronization block is doing nor why it is necessary.
Can someone explain in a "synchronizing for dummies" kind of way?
In a book I am reading, the author tells me "The synchronization is necessary, since the members we manipulate within the
synchronized block could be manipulated in the onPause() method on the UI thread."
He creates an Object named stateChanged and instantiates it as a new object.
Then, in the synchronization block he uses the stateChanged object as the argument.
This whole thing is throwing me off and I do not like to move on until I have a pretty good understanding of what is going on.
The classic example is: Imagine you have two threads of operation, and both of them reference the same method:
public void addToGlobalVar(int y) {
int x = globalVar; //what if a thread stops right after this line?
x += y;
globalVar = y;
}
where globalVar is some other predefined number that this method can interact with and set. Lets say globalVar is 50.
Threads get computing time on a somewhat arbitrary basis, so you never fully know the precise nanosecond one stops and the other gets CPU time.
In this example, if you launched an AsyncTask in addition to the UI thread, and both at some point use addToGlobalVar(10), what can happen is that one thread might be interrupted at line 2 of that code block. If the other thread goes through while that one is sleeping, it will successfully set globalVar to 60. But when the other one wakes up, it still thinks x = 50, and its going to then set it to 60. So in essence you just made 50+10+10 = 60. Hopefully you can see how this becomes a problem.
You can fix this simple example by making the calculation atomic (skip declaring x, 1 line, all calcs done) or if the logic wasn't able to be condensed to 1 line, you make a block of code atomic by using synchronized.
The book to read is Java Concurrency in Practice.
You should really just segregate this idea from Android, although your code is going to be running on Dalvik this is a Java concept. Not an Android one.
The synchronized block takes an object as a parameter, any object, and when flow enters the body of the synchronized block, any other thread that runs in to a synchronized block with the same instance (object) as the parameter has to wait for the previous one to complete. That's a very basic description.
This is an entire sub-field of computer science and without serious study you will probably not understand it.
You have to fully understand it before you use it. It is standard android synchronization using object-oriented monitors. You have to understand it to write multi-threaded programs, however it is somehow dated (better use java.util.concurrent for anything thread/synchronisation related instead).
Anyhow - you need to know what it is about - read the related java tutorial part:
http://download.oracle.com/javase/tutorial/essential/concurrency/sync.html