Develop Reference

putFragment of FragmentManager generates same integer for all fragments

I tried to use putFragment to save reference for fragments for using it in future (and not recreate) before replace.
BaseFragment last = (BaseFragment) mContext.getSupportFragmentManager().getFragments().get(mContext.getSupportFragmentManager().getFragments().size() - 1);
mContext.getSupportFragmentManager().putFragment(mContext.getBundle(), last.getType().toString(), last);
And before creating new fragment i check bundle for fragment existing:
BaseFragment f = (BaseFragment) getSupportFragmentManager().getFragment( mBundle, type.toString());
if(f != null)
return f;
else
// create new fragment
FragmentType is just my enum:
public static enum FragmentType{
PROJECTS,
BALANCE
}
But for all fragments (for all keys in bundle) it generates same integer value.
So getFragment method returns wrong fragment. Where is the problem?
I saw this post with same issue. But it still is not resolved...
I needed it for storing fragment state while replacing it. I tried it after this answer.

Use cases of these methods are described in this question: get/putFragment()
Also, don't use getFragments(); It is a method annotated by #Hide and is not supposed to be used, because android can destroy an recreate activities and fragments at any given time, so the list is not always correct.
While the methods do give acces to the 'pointer' of the fragment, It is meant to only restore state.
Java is a language with a garbage collector, and android builds on top of that with objects that do not give us control over their lifecycle.

These methods (put, get) are meant to be used inside onSaveInatanceState and onRestoreInstanceState to save the "state" of the fragment so that it (the state) can be restored later.
Are you using them in that context?
If not, you are essentially trying to override the way the OS handles the fragments' lifecycle, which is asking for trouble.
In other words, don't try to get back the same instance, the system just cannot guarantee that to you.
There is an interesting discussion on this issue over at the android developers

How do Android Handle Fragment livecycle within a ViewPager?

This is not going to be a straightforward question. I'll explain myself as best as I can, so this might be a little bit long.
Explanation
Recently, I've been fighting a lot with Fragments and ViewPager.
What I found out, is that ViewPager, holds as many fragments as you want. When it's instantiated for first time, depending on the type ( FragmentStatePagerAdapter vs FragmentPagerAdapter Difference between FragmentPagerAdapter and FragmentStatePagerAdapter), it's going to create all Fragments or just the "needed" ones.
In those fragments, you're free to use methods like getActivity() which basically is the context used for creating Views, finding Views, displaying dialogs...
However, as soon as you leave the fragment, the getActivity() starts returning null, because the fragment is dettached from the Activity. Sounds logic.
But, what sounds illogical to me, hence im asking here, is that when you go back to the fragment page (or close enough), it's again triggering the base methods, like onAttach.
As soon as you've onAttach, you can keep the context provided by Activity, but if you try to do something like the following (pseudo-code):
class MyFragment extends Fragment
{
private Context context;
private doThingsAfterAttach()
{
getActivity(); //this is null.
null != context; //TRUE
}
#Override
public onAttach( Activity activity )
{
context = activity;
doThingsAfterAttach();
}
}
What you can see, is that getActivity() is returning null, even though you're calling this method after onAttach is triggered.
If you decide to cast the kept context to an Activity, to perform findViewById tasks, you will see that the view you're trying to find, is null, which means that couldn't be found / doesn't exist.
Issue
Imagine you have a ViewPager with 5 tabs (fragments).
When a task is performed on ANY of those 5 tabs, you want to notify the "holder" activity, in order to notify all fragments that should update it's content, as something has changed.
But when you notify them, if they have to change layout, they can't do it, because as soon as you try to findViewById, two things can happen: getActivity() is returning null, hence you can't get Views, and if you cast context to activity, it won't return any View when you search for any.
What scares me the most, is that when you rotate the device, acts like leaving the fragment page and going back; "loses" activity.
Real question
So what I'm looking for, is an answer which explains me what's happening internally, so I can find the appropriate code to handle those situations.
There's not much code I can provide, because it would be useless. Whoever uses ViewPager with Fragments, probably handled with those things, so you will understand me.
Thank you, and here I am ready to answer your questions.
Thanks.

note: This is a lengthy and probably boring post
Your problem with the ViewPager is not new. Don't feel bad. We've all went through what appeared to be a magic solution to a new paradigm (and I will put new in Italics because it wasn't necessarily new at the time). With the added support for Fragments (through the State and Stateless adapters that you mention) things quickly became more interesting.
I'm not going to go into the details whether the adapters are good or annoying (you can draw your own conclusions) or why the adapters (and the widget itself) lack very basic stuff that nobody understands what where the Android developers thinking when they exposed the public API for these.
Instead, I'm going to help you manage Activity/Fragment communication the way -I think- everybody does.
What is happening with Fragments in a ViewPager?
The Activity/Fragment concept is -in my humble opinion- horrible and it was a hack. But an effective one, since it quickly proved that it worked, to the point where the Android team decided to start adding Fragment support to more and more stuff. Even nested Fragments, yes Fragments inside Fragments! And since Android developers (less and less but still very often) have to deal with old versions of the platform, then all this support was initially added to the Support Library, and it never moved out of it.
But let's get back on topic. The Fragment lifecycle is somewhat confusing (in part thanks to the poor naming of some lifecycle methods and the order they are -not- guaranteed to occur). Some things are obscurely hidden behind unfriendly callbacks (TreeLayout anyone?).
So… the ViewPager needs an adapter. The adapter is in charge of supplying the ViewPager with its views. So while the ViewPager is a View that understands what a touch, drag, fling, draw, measure, etc. is, it really expects an Adapter to provide the data to display. (This is a huge simplification).
And here we have Two types of adapters which know how to deal with Fragments. One maintains a state and the other doesn't. Meaning one doesn't really release anything (FragmentPagerAdapter) and one does indeed release its fragments (FragmentStatePagerAdapter)… but wait… what is "release" in this context?
Turns out that Fragments don't really exist in a free world, they are dominated by a FragmentManager, who makes sure that they are not late and decides when to release them. This is a rule that not even the Adapters can override! So they must report to this FragmentManager anyway. You can tell that the adapter must talk to this FragmentManager because it must receive one in the Constructor!
public static class MyAdapter extends FragmentStatePagerAdapter {
public MyAdapter(FragmentManager fm) {
super(fm);
}
Who is this FragmentManager anyway?
Well, the documentation is not very friendly, it just says:
Interface for interacting with Fragment objects inside of an Activity
Ok, thank you Mr. obvious!
In practice, the FragmentManager is a static class, you don't create it, you just get it from an Activity.
Now think of the FragmentManager of a controller who keeps a reference to your fragments, even long after you've left them. It knows how to bring them back, it helps them save their state when configuration changes (rotations, for example), it keeps a pile/stack of them, knowing in which order they can be popped back to life and it does all this, sporting FragmentTransactions that must be committed, much like a relational database transaction.
Most of this complexity (and I'm sure they had their reasons) happens behind the scenes, so you don't really have to worry too much about it.
Can we go back to the original question now?
Yes yes… it's a long subject as you can see, but let's get to the point, because I have to go back to work…
When the Fragment gets out of view, the adapter secretly tells the FragmentManager: "Yo dawg, this Fragment is no longer needed here -for now-". (It may not use that phrase but something like that). For a more detailed response, you can actually look at the more or less recent code of the FragmentStatePagerAdapter and learn a lot from it.
Look how it has a list of Fragments and SavedStates:
private ArrayList<Fragment.SavedState> mSavedState = new ArrayList<Fragment.SavedState>();
private ArrayList<Fragment> mFragments = new ArrayList<Fragment>();
and a reference to a FragmentManager:
private final FragmentManager mFragmentManager;
Not surprisingly, FragmentPagerAdapter doesn't have a list of Fragments nor a list of saved States, it simply lets the FragmentManager do its job.
So let's first look at the "State" pager adapter to see what's doing when it's time to Destroy a Fragment…
With Google's permission, let's look at the source code for destroyItem():
1 #Override
2 public void destroyItem(ViewGroup container, int position, Object object) {
3 Fragment fragment = (Fragment)object;
4 if (mCurTransaction == null) {
5 mCurTransaction = mFragmentManager.beginTransaction();
6 }
7 if (DEBUG) Log.v(TAG, "Removing item #" + position + ": f=" + object
8 + " v=" + ((Fragment)object).getView());
9 while (mSavedState.size() <= position) {
10 mSavedState.add(null);
11 }
12 mSavedState.set(position, mFragmentManager.saveFragmentInstanceState(fragment));
13 mFragments.set(position, null);
14 mCurTransaction.remove(fragment);
15 }
Line 4 starts a FragmentTransaction if one hasn't already been started.
Lines 9-11 pads out the mSavedState array with null entries until it’s at least the size of the index of the fragment we’re removing.
Line 12 saves the state of the Fragment being removed (so it can be restored in the future if needed).
Line 13 is effectively removing the Fragment reference…
Line 14 adds this to the Fragment's Manager transaction so the FragmentManager knows what to do.
(Note: there's an obscure bug in the ViewPager when you add/change/remove Fragments dynamically, I'll link to the problem/solution at the end).
When you add a Fragment to the ViewPager, the process is relatively similar (see instantiateItem() method…
It first checks if the object is already instantiated, then it returns it immediately.
If the Fragment is not there, one is created…
if (mCurTransaction == null) {
mCurTransaction = mFragmentManager.beginTransaction();
}
Fragment fragment = getItem(position);
Remember you extend this Adapter and create your own getItem() method, this is where it gets called. You are giving the Adapter a newly created Fragment.
Next the Adapter checks for the savedState of the fragment to see if it can find one (and here it makes a mistake) (see link at the end)…
Finally it proceeds to add the newly received Fragment:
while (mFragments.size() <= position) {
mFragments.add(null);
}
fragment.setMenuVisibility(false);
fragment.setUserVisibleHint(false);
mFragments.set(position, fragment);
mCurTransaction.add(container.getId(), fragment);
It must add null padding for the array to be the exact size, and also uses the FragmentTransaction of course.
The moral of the story so far is that the Adapter keeps its own collection of stuff but it keeps the boss (a.k.a.: FragmentManager) happy by letting him know he's in control.
For reference, the support v13 versions are pretty much the same, but have references to the non support version of Fragment, FragmentManager, FragmentTransaction, etc.
So if the Adapters either keep a list and ask the FragmentManager (through a FragmentTransaction) or just use the FragmentManager, what does the FragmentManger do?!
This is lot more "complicated" but the FragmentManager implementation has a list of Added and Active fragments (along with a myriad of other collections and data structures).
ArrayList<Fragment> mActive;
ArrayList<Fragment> mAdded;
^ That's coming from the FragmentManagerImpl class!
So I won't go into details about the FragmentManager (you can find its source code here), because it's a really big class and it uses transactions for everything and it's super messy. It keeps a state machine about each fragment (created, initializing, etc.). The interesting method is perhaps moveToState() and this is where the Fragment lifecycle callbacks are made, so take a look at the source code to see what's going on.
Also take a look at the removeFragment() method there, which ends up calling the moveToState() method in the end.
Enough with all this… when can I call getActivity() and not get null in my Fragments?
Ok, so you provided an example of what you wanted to do.
You have ViewPager with 5 Fragments (for example) and you want to notify them that something has happened so they can do something about it. A lot of somethings.
I'd say that this is a typical Observer pattern + ViewPager.OnPageChangeListener().
The observer pattern is simple, you create an Interface like:
public interface FragmentInterface {
void onBecameVisible();
}
Your Fragments implement this interface…
public Fragment YourFragment implements FragmentInterface {
and then you have a:
#Override
public void onBecameVisible() {
if (getActivity() != null && !getActivity().isFinishing()) {
// do something with your Activity -which should also implement an interface!-
}
}
And who calls onBecameVisible?
Your ViewPager.OnPageChangeListener() in your Activity:
public void onPageSelected(final int i) {
FragmentInterface fragment = (FragmentInterface) mPagerAdapter.instantiateItem(mViewPager, i);
if (fragment != null) {
fragment.onBecameVisible();
}
}
So now the Activity can reliably tell the Fragment that it has become visible.
Now… if you want to tell the OTHER Fragments, then you have to have a list of "listeners" (Observer pattern) but you must understand that according to what we've seen about the Adapters and the Fragment Manager, Fragments may be detached (even tho they may not necessarily be destroyed). The fact that they are not visible, could mean they are prone to have their view hierarchy destroyed (so changing it at that point would not be possible).
Your best bet is to make sure your onCreateView is capable of asking which view should be displayed based upon your business logic.
The best way to understand the lifecycle is to add a Log to EACH of the Fragment lifecycle (onStart, Stop, Pause, Resume, Detach, Destroy, ActivityCreated, etc…) and see how they react when you move through the ViewPager.
I hope this lengthy post gives you an idea about what's the ViewPager doing with your Fragments. Let us know if you have more specific questions or if I haven't really helped you :)
Regarding the BUG I mentioned earlier, take a look at this post which talks about a bug in (and a fix for) the way FragmentStatePagerAdapter handles fragment restoration.

I think getActivity() return null onAttach() because you haven't called it's super in other words it didn't get the chance to set it as its activity . Also you don't need to call getActivity() to find views, you can keep their references on onCreateView() and update them on onStart() if there should be any change in their values.
Also keeping the activity context as global will prevent the fragment from being garbage collected as long as activity lives. So release the reference on onDetach().

If your problem is that you want to notify the Activity, simply pass the Activity as "listener" to the fragment. Remember to pass it as WeakReg tough to try to avoid leak issues.

Android - fragments

I'm starting using fragments in android but I have doubts: 1.- is necessary using this part of code (newInstance, setArguments)?, if so, why?
public static class DetailsFragment extends Fragment {
/**
* Create a new instance of DetailsFragment, initialized to
* show the text at 'index'.
*/
public static DetailsFragment newInstance(int index) {
DetailsFragment f = new DetailsFragment();
// Supply index input as an argument.
Bundle args = new Bundle();
args.putInt("index", index);
f.setArguments(args);
return f;
}
public int getShownIndex() {
return getArguments().getInt("index", 0);
}
http://developer.android.com/reference/android/os/Bundle.html
2.-if this code would have an OnCreate method, when will this be called?

1) Why is newInstance needed?
newInstance methods are needed because the framework encourages you to only ever have one constructor, taking no arguments. Having more than one constructor is somewhat frowned upon.
It needs this constructor to be public, so that it can recreate you Fragments when, for example, you undergo a configuration change. The same Bundle that you passed to setArguments on the original will be given to the copy too (as well as a savedInstanceState Bundle from onSaveInstanceState).
Doing this for all fragments, including ones that have no arguments leads to consistent usage of fragments and ease of maintenance. Namely, do make sure to always pass an arguments Bundle, even if it's empty, so that you don't have to check getArguments() for null.
2) onCreate in the Fragment lifecycle.
The Fragment lifecycle is officially called "nested" but when adding new fragments I like to think of it as "playing catch up". For example, if your Activity has started (has had its onStart called), your Fragment will get a rapid succession of onAttach, onCreate, onActivityCreated, onCreateView, onStart in essence catching up to the Activity's lifecycle. You can enable debugging with FragmentManager.enableDebugLogging(true); to track the lifecycle changes - look for moveto (forward direction) and movefrom (rewinding the cycle - destroying/detaching the fragment).
That said, onCreate is kinda easy to predict - since you're probably first creating the fragment after onCreate of the Activity, it would be called almost immediately when the transaction executes.
However, if you undergo a configuration change, the Fragments are destroyed with the old Activity and then created before the new Activity (it happens in that super.onCreate(savedInstanceState) call that you have in your Activity) - onCreate of all the fragments will be called before the Activity's onCreate returns. Once it returns, the Fragments' onActivityCreated will be called and from then on the fragments and activity will move in lockstep. This is all unless you have setRetainInstance(true), of course.
I hope this clears it up. I strongly suggest enabling debugging and just playing around with the fragments to get a better feeling for their lifecycles.

If you want to pass arguments to a fragment, the new instance method is necessary.
You pass arguments this way, because fragments can be in a detached state, and when they are re added may need to be reinstantiated. Android will then call the newInstance method again with the same arguments, and your fragment will work correctly.
If you passed your arguments to an onCreate method (which is called only when the fragment is first created), then when a fragment is reinstantiated, it wouldn't get those arguments.

Saving state on List Data when App Resumes

Here is my set up.
I have a Main SherlockFragmentActivity. It swaps many ListFragments back and forth with FragmentTransaction's. To indicate loading, anytime a ListFragment loads the data I call:
setSupportProgressBarIndeterminateVisibility(true);
The problem:
When the main Activity mentioned above first starts, or the user leaves and goes to other apps and then restarts this one after an extended period of time, the SherlockFragmentActivity seemingly reloads, there is no progress dialog in the ActionBar, the screen is white for a few seconds, and then the list data repairs (The length depends on the data connection).
Here is some supplemental code: When the main/base Activity first loads, this is one of the first things I do in the onCreate():
// Set up Main Screen
FragmentTransaction t2 = this.getSupportFragmentManager().beginTransaction();
SherlockListFragment mainFrag = new FollowingFragment();
t2.replace(R.id.main_frag, mainFrag);
t2.commit();
FollowingFragment is the one that will always load in this instance. It contains a ListView and an AsyncTask pulling data from a MySQL database.
My question: How do I prevent this delay? And how do I handle maintaining the data when user leaves for extended periods of time?

This is the normal behavior, it happens because your activity has been killed to save memory for other apps, when your app was in the background. And when it comes back to the foreground, the system recreate your activity, which will recreate your fragment.
But if your really want to avoid recreating your fragment, you can use setRetainInstance in your fragment's onCreate method:
public void setRetainInstance (boolean retain)
Control whether a fragment instance is retained across Activity re-creation (such as from a configuration change). This can only be used with fragments not in the back stack. If set, the fragment lifecycle will be slightly different when an activity is recreated:
onDestroy() will not be called (but onDetach() still will be, because the fragment is being detached from its current activity).
onCreate(Bundle) will not be called since the fragment is not being re-created.
onAttach(Activity) and onActivityCreated(Bundle) will still be called.
And use something like this in your FragmentActivity's onActivityCreated method:
#Override
public void onActivityCreated(Bundle savedInstanceState) {
super.onActivityCreated(savedInstanceState);
FragmentManager fm = getFragmentManager();
// Check to see if we have retained the worker fragment.
mRetainableFragment = (RetainedFragment)fm.findFragmentByTag("fragmentTag");
// If not retained (or first time running), we need to create it.
if (mRetainableFragment == null) {
mRetainableFragment = new RetainedFragment();
// Tell it who it is working with.
mRetainableFragment.setTargetFragment(this, 0);
fm.beginTransaction().add(mRetainableFragment, "fragmentTag").commit();
}
}
But be aware that, this should only be use for headless fragment (fragment without UI, i.e return null in onCreateView, aka worker fragment). You can still use this method for UI fragment though but it is not recommanded by google, in that case the data must be stored as member (field) in your activity. If the data which should be stored is supported by the Bundle class, you can use the onSaveInstanceState() method to place the data in the Bundle, and retrieve that data in the onActivityCreated() method.
Moreover this only works if the fragments is not added to the backstack.

According to the Android developer reference page on Activity, you have to request the progress bar feature before calling setSupportProgressBarIndeterminateVisibility():
requestWindowFeature(Window.FEATURE_INDETERMINATE_PROGRESS);
setContentView(R.layout.activity_main);
setSupportProgressBarIndeterminateVisibility(true);
The other issue, reloading the fragments, is due to Android killing your ListFragment so that they have to reload could be resolved by overriding onSaveInstanceState(Bundle outState) and caching your data there to be retrieved in your ListFragment:
#Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
if(savedInstanceState != null) {
// retrieve data from Bundle here
} else {
// no data, we need to reload from network
}
// initialize your View here
}
This method is not guaranteed to run all the time, however (it's not in the Fragment lifecycle). As such, you should also make sure you cache the data in onPause() and use it instead of always loading from a network connection.
#Override
public void onPause() {
super.onPause();
SharedPreferences prefs = getActivity().getSharedPreferences();
SharedPreferences.Editor editor = prefs.edit();
// put your data here using editor
editor.commit();
}
Then you can load this data in your onCreateView() by retrieving an instance of SharedPreferences and using prefs.getString(String key) and other methods.

When your app get killed, you lose your activity state and data! There are two scenarios that I can assume about your AsyncTask:
1. you are pulling some data from a Webserver. In this case I personally think caching your data which you retrieved from webserver is a better solution than implementing serializable.
2. You are pulling lots of data from local database (which causes retrieving data to take some time). In this scenario I suggest retrieving only as much data as you need, not more! (for example you can retrieve 20 items, and when user scrolling to the end of ListView retrieve next 20 items).
This solution helps your application retrieve data faster.
PS: To give you a clue how to implement the WebserviceModule with cache capability, which I assume is located in your AsyncTask, you can save every response from webserver in the SDCard and every time you trying to retrieve some resource from webserver, you should check the SDCard to see if your request already sent and cached! For every request, your should make a unique signature base on url and post parameters to recognize cached files.

When you return to activity after extending period of time, the whole app being restarted. So You can't rely on object variables to save data.
So You could avoid delay You've mentioned with saving data to some local storage in activity onStop() method. For example, shared preferences.
And when You call onCreate(), check whether You have data saved and use it if exists (and clean up to have "clean" start next time), otherwise start asynctask.

ASyncTask, hidden fragments, retaining instances, and screen orientation changes

My setup is as follows.
I have a FragmentPagerAdapter called from my Activity which loads two fragments. This is setup within onCreate.
In onResume I call an ASyncTask which loads data from a database, and then calls a callback in my activity onLoadComplete via a load data listener.
#Override
public void onLoadComplete(JSONArray data) {
// TODO Auto-generated method stub
LocalFragment fragmentB = (LocalFragment)getSupportFragmentManager().findFragmentByTag(ListTag);
fragmentB.setList(data);
LMapFragment fragmentA = (LMapFragment)getSupportFragmentManager().findFragmentByTag(MapTag);
GoogleMap our_map = fragmentA.getMap();
fragmentA.plotP(myLocation,data);
}
The fragments are initialized by the Pager, and within each fragments code I set the respective tag e.g in LocalFragment
#Override
public void onAttach(Activity activity) {
// TODO Auto-generated method stub
super.onAttach(activity);
String myTag = getTag();
((PagerTest) activity).setListTag(myTag);
Log.d("what",myTag);
}
This allows me to access the fragment, call a function within it which populates a list or populates a map. It works.
What I am now trying to do is account for screen orientation changes.. If while the ASyncTask is running the orientation is changed, the app crashes.
As suggested here: Hidden Fragments I have been trying to implement a hidden fragment which saves the state of my ASyncTask. So what I have done is set it up so in onResume of my Activity i call a function
static LoadDataFromURL the_data = null;
static JSONArray pub_data = null;
private static final String TAG = "RetainFragment";
public RetainFragment() {}
public static RetainFragment findOrCreateRetainFragment(FragmentManager fm) {
RetainFragment fragment = (RetainFragment) fm.findFragmentByTag(TAG);
if (fragment == null) {
fragment = new RetainFragment();
fm.beginTransaction().add(fragment, TAG).commit(); // add this
}
return fragment;
}
which essentially saves my data.
Basically what this means is that if i rotate my screen i dont call my ASyncTask again.. the screen just rotates.. it works perfectly.
If however I go back to the main menu and then click on the activity again the screen returns blank (but does not crash). My understanding is that the data is retained as an object in the fragment, but on reloading the activity afresh the data needs to be set again.. I.E onLoadComplete needs to be called to populate the list/map..
So i concluded that if initially after the ASyncTask completes i save the returned data in my hidden fragment onRetainInstance, then i could simply call onLoadComplete and pass it..
The problem is, in this situation seemingly the fragment has not been called yet, as such the tags are null, and calling the callbacks within onLoadComplete crashes the app.
I have been banging my head over this for ages.
My ASyncTask is in a seperate class: LoadDataFromURL
What i want to achieve is as follows - a fragmentviewpager whereby on screen rotate the ASyncTask is retained on rotate/attached to the new activity, and if it has completed before it shouldn't be run again..
Could anyone advise.
Many Thanks
EDIT
Having changed the variables in my secret fragment to public variables, everything has seemingly come together.. BUT because im not 100% sure how/when things are called, I dont fully understand WHY it works..
So.. I call findOrCreateRetainFragment and it either creates a new 'secret' fragment or returns the current instance.
If it is returning a current instance, i dont call my async task again. If it is not, I call my asynctask and load the data.
With this setup, when i load the activity and rotate the screen, it rotates as expected woop.
Now, when i go back to the main menu and then click the activity again, it calls the async task.
My understanding is that on rotate the async task is not called again, and the viewpager is somehow saving the fragments.
On the other hand, when i go back my activity is destroyed, as is my secret fragment, and as such when i click on it again it loads the data. THis is essentially what i want..
Have i understood this correctly?
Thanks

There are a few issues here that you're experiencing (I think).
First of all, the reason your callbacks crash is because they're attached to an old Activity that no longer "exists" after a screen orientation and/or Activity push. If you use onAttach() to attach a callback to your fragment, you must use onDetach() to detach that callback when the Fragment is removed from the Activity. Then, whenever you call the callback, check for a null so you don't send data to a dead object.
Basically, the logic you're trying to use here is:
Start Activity.
Check if your Fragment exists. If it does, grab it. Else, create it.
Retrieve the data if it exists. If not, wait for the callback.
Because of the nature of callbacks (depending on your implementation), you will not receive data until the event fires. However, if the Activity is gone and the event has already fired, the callback won't execute. Thus, you have to retrieve the data manually. When using setRetainInstance(), it's helpful to think of it as this entity detatched from your Activity. It will exist as long as you don't pop the current Activity or push a new Activity. However, your current Activity will be destroyed upon screen orientation changes while the Fragment won't. As such, the Fragment shouldn't rely on the existence of the Activity.
A much more elegant solution to the problem that you may want to look in to is implementing the Android Loader API. Loaders are handy tools that are handled by the system that work is roughly the same way but are more in-tune with asynchronously retrieving data. They work effectively the same way. You simply start your loader and the system with either create one if it doesn't exist or re-use one that already exists. It will remain in the system by the LoaderManager upon configuration changes.
EDIT:
To answer your edit, I guess I'll explain what's happening. It's convoluted, so just tell me if anything needs clarification.
Fragments aren't technically speaking part of your currently running Activity. When you create an instance of the Fragment, you have to call beginTransation() and commit() on the FragmentManager. The FragmentManager is a singleton that exists within the realm of your application. The FragmentManager holds on to the instance of the Fragment for you. The FragmentManager then attaches the Fragment to your Activity (see onAttach()). The Fragment then exists within the FragmentManager which is why you never really have to hold a reference to it within your application. You can just call findFragmentByTag/Id() to retrieve it.
Under normal circumstances, when your Activity is being destroyed, the FragmentManager will detach the instance of your Fragment (see onDetach()) and just let it go. The Java garbage collect will detect that no reference to your Fragment exists and will clean it up.
When you call setRetainInstace(), you're telling the FragmentManager to hold on to it. Thus, when your Activity is being destroyed on a configuration change, the FragmentManager will hold on to the reference of your Fragment. Thus when your Activity is rebuilt, you can call findFragmentByTag/Id() to retrieve the last instance. So long as it didn't keep any context of the last Activity, there shouldn't be any problems.
Traditionally, one would use it to keep references to long standing data (as you are) or to keep connection sockets open so a phone flip doesn't delete it.
Your ViewPager has nothing to do with this. How it retrieves the Fragments is completely dependent on how you implement that Adapter that it's attached to. Usually, retained Fragments don't have Views themselves because Views hold Context data of the Activity they were created in. You would just basically want to make it a data bucket to hold on to the data for the Views to pull from when they're being inflated.