Live Q&A discussion thread - March 14th

RUBBISH

Is this caused by the datacentre move 31-Jan-2011?

Commission refund a good gesture as compensation for Saturdays outage , however it must be done on enough markets .

Pretty much a rehash of the last Q&A so far..

It looks as though they are going to offer commission-free markets for the Cheltenham Festival, cricket World Cup, Champions League etc.

If there's no catch, this is a handsome offer!

The selection of the commission refund markets is quite broad and a welcome step, and appears to be set to compensate in those sports which were most effected, which is, of course, fair.

However, a significant portion of Saturday night's betting on the World Golf Championship was lost, much to the frustration and financial loss of golf bettors, yet there's no compensation for this.  Golf might not be the biggest sport on BF, but there were a lot of frustrated punters left high and dry on a significant golf event.

Ah, it is only one race a day. Still, not bad....

The Supreme Nov. Hdl wd have been too much to hope for.

As stated on another thread, give everyone who has placed a bet on the site in march £50 free, no strings attached. A few Commission free markets means virtually nothing tbh.[>o]

Very short notice of this forum chat - you've only allowed two hours for questions.

As to the "compensation".  It's farcical - what if we don't bet on the markets?  We shouldn't be forced to spend money just to qualify for "compensation".

The gall of you people to tell us that you have sufficient management structure and good IT practices in place.  You clearly don't, else you wouldn't need to roll back your upgrades with such regularity.

Well, Betfair's IT manager IT believes Betfair's IT management is good enough.  That's a relief.

no horse racing 100% commision free market on Gold Cup day -  come on Betfair look again please.

Sorry the Triumph is covered, my mistake.

re: Please can you tell me how many test environments there are at Betfair and what methodology is used in terms of stress & contention testing to migrate system upgrades?
We have a huge range of test environments including development, performance/stress and staging. Even though our Quality Assurance team of over 100 people undertakes extensive testing, it is very difficult to replicate the actual live behaviour seen on the site.

It is not "very difficult to replicate the actual live behaviour" because you run historical data against a sandbox - just as every other company with a decent IT department does.
Your 100 QA staff can't even script for the most basic interface checks - take the error of missing markets that crops up time after time.  Do you run any UAT - because I'm pretty sure even the most novice of users would be able to spot a gaping hole down the left-hand side of the screen!

A good question yeahyeahwhatever.

I hope it's asked?

re: The last time the system crashed on a Saturday afternoon, you did some testing on the following Thursday. This involved stopping the live system, which caused API bots trading systems to crash. Will you be doing this again?
The last time we did this we were trying to check whether the fix we had in place would be completely effective under live conditions. As this particular problem is a completely different issue that we know we have fixed, we have no immediate plans to do this again.

So you DO test against a live system then... jesus wept BF, are you trying to lose all credibility.
A company with so much spare server capacity that they can (alledgedly) triple their storage and still have lots to spare!

... but not enough spare storage to set up a separate test environment...

those comm free markets will have been very carefully selected to ensure they minimise their losses.  They want to ensure they call recover the comm loss by offsetting elsewhere.

dear betfair

if i dont place a winning bet, when the 100% commission back comes into effect, how will i feel compensated?

Can you please be more explicit regarding testing, around which there is much scepticism.  For example can you please answer the question raised above: - how do you do User Acceptance Testing (UAT)?

Betfair
I agree with what has been said about golf bettors etc. some people may not want to bet on the commission free markets offered this week! Anyway why should we HAVE to place bets "risking" more money in order to qualify (if the bet actually wins)for compensation which is a very small percentage of what I and many others would have lost at the weekend. For traders like me, the majority of liquidity is provided by traders and by offering 5% of winnings as compensation I simply do not think this is good enough.
Alex

this doesn't make sense:

"We are confident that we have the right people, process and technology to ensure both reliable service delivery and change.  Our stability issues over the past weeks have shown that we have had our mix wrong in terms of the type and velocity of change we've put in front of our customers"

it is unquestionably impossible to be confident that you have the right processes to ensure reliable service delivery and change, given the admission that you've had your mix wrong and have only found this out through numerous site crashes.

Re this in the Q and A:
"Why aren’t all channels taken down during an outage to prevent bets from being unfairly matched (i.e. API, Mobile, etc)?
As soon as we become aware of a significant issue, it is our policy to disable all betting channels to ensure a level playing field for customers."
Please if you manually disable things (a good thing) but we still have access, PLEASE make it so markets are suspended but bets can at least be VIEWED and CANCELLED. If you can't cancel them, then it comes back up later especially if you're now away, you're obviously possibly in a bad scenario, also if you can see what bets you have, that's obviously extremely useful as well at the time.

betfair: so anyone who got stuck in a large losing position on saturday and has had to scale down or out as a result will get compensated less/not at all. whereas anyone who got stuck in a large winning position will be able to bet more and thus benefit more.

who came up with a compensation idea that discriminates against those that lost the most? you should be refunding people commission from last week, not this week. and implementing a discount decay suspension.

there might be no crashes this week, but i dont have much confidence that when they try to roll out new "features" they wont introduce something catastrophic to performance. They have had 10 years of website experience but test environments still dont match live

Aaarrrggghhhhh!

They make me so angry!

I lost 2.2k on Saturday and in return they're going to give me my commission back on the 2 markets I intend to trade on that list!!!!!!!

What a pile of cr*p

I traded out on **** but that's not the point

generally a very disappointing q&a session so far. not addressed the issues of anyone trading. not restored my confidence at all. whilst traders seem to have been considered pretty unimportant due to low comparative commission for betfair, maybe the importance of them in providing liquidity for others to bet against is now being understood. recent weak markets will continue if their issues are not addressed. if someone has a £100k trading bank for example that he is prepared to risk say 5% of in getting the trade right or wrong how do you expect him to react now he is expected to risk the whole £100k of it?

I have a quick question:


In future, would it be possible to warn of a planned outage using the little speech bubble in the top right?

we identified that the database had some threads that were frozen and we needed to free them up in order to guarantee performance for Sunday

So thats 2 issues a web front end issue and now a DB issue

We have also imposed a change freeze for the next 7 days so we can maximise the stability of the site.

Could you start warning us in advance, say a day, of when changes might be thrust upon us?

Barring that, could you try doing them when fewer major events are on , rather than during england cricket matches or saturday football?

We intend to keep adding great new products

I suspect I speak for 90+% of the forum when I say simply
"Please don't"

are any betfair employees actually going to reply on this thread to us or are we ranting at ourselves????!!!!

imho there will always be problems in february to the cheltenham festival due to doing stuff to try and make cheltenham go smoothly which causes site probs. this year has been exceptionally bad though. agree all the new 'rubbish' should be on a separate site as the slowness now must seriously put off anyone interested in having a traditional bet. personall i lost £600 a few weekends earlier so have taken a couple of weeks off expecting saturday to happen some time soon. the question is now do i go back at all? the question of providing compensation via commission free markets is bizarre as it just seems like a promotion open to everyone rather than adressing anyone's losses. very sad to see a company i once considered the very very best of british come to this.

you watch them pimp the zero commission markets for all they're worth in the post as well.
sickening

well said Lori

alexpollard

We have been trying to grab questions raised in this thread and dealing with them on the Live Q&A thread.

Sorry if that's been confusing.

Betfair Customer Services     13 Mar 11 16:10 

When the website failed on Saturday, our first step was to disable Betfair for all our customers on the web, API and mobile services.  Once we identified the actual problem, we determined that we needed our website "available" but with betting disallowed.  We recovered the site internally around 18:00 and re-enabled betting as of 20:00 once we were certain it was stable.


So 24 hours later you release your apology, even then you get it wrong, i had a load of bets matched from 17.02 to 17.17
probably not the biggest issue but does show a level of incompetence to not get the facts right.

i put in a question & yet again no answer.
this site has become a nightmare & not because of just saturday. EVERYDAY is a nightmare.
i have to log in & out 10 times a day, its takes 5 minutes to get where you want to go when you log in. its slower than any site ive EVER visited.

does anyone at betfair really know how bad this site is???
theres way too many issues to just fix. the site was so much better when it started, everything is sooo slow.

IE is now unusable & firefox aint much better, ive even bought a new pc cos i thought that was the problem. made no difference what so ever.

So anyone who got stuck in a large losing position on Saturday and has had to scale down or out as a result will get compensated less/not at all. Whereas anyone who got stuck in a large winning position will be able to bet more and thus benefit more. Who came up with a compensation idea that discriminates against those that lost the most? You should be refunding people commission from last week, not this week, and implementing a discount decay suspension.

Betfair: We looked at a whole range of options but we felt the fairest option was to give a commission refund and announce it in advance so that everyone has the chance to benefit.

--------------

how does anyone who got wiped out have a chance to benefit? and again, what is the rationale behind a "compensation" system that people can benefit from in inverse proportion to the amount they suffered?

re: As I can see from your explanation on the core threading issues you have encountered working in a similar environment (High Load server site) I want to congratulate you on the quick decision not to restart betting after the issues you encountered. I believe the decision actually saved your customers form further issues on Saturday.

UNTRUE.  I placed a bet (and it was matched, account balance updated) at 17:19 on Saturday just to see if the site was really back.  If you read the horse racing forum you'll find many others who were also able to place bets, only to the market then remain suspended. 

Congratulations and BF are not words that go together of late.

Betfair you are a total joke. Hope you lose a load of customers, you deserve to with your arrogance. It's quite clearly you couldn't give a stuff with your pathetic response. Give people proper compensation or don't waste their time!

i personally use betfair rapid to avoid the slowness and the 'garbage' on the main site. whilst there can be problems with it like anything else (that could cost you money) i'd thoroughly recommend giving it a try (to technophobes) as it's easy to use and avoids the slowness of the main site which i'd agree with the 'uselessness' comments already made and outlined by others.

some constructiveness by me at last lol

nice one woo i will try it, but thats not the point.

why is betfairs site the slowest site on the internet? they need to sort it out its not getting better its getting worse.

Hope the CTO liked his experience of the the new forum's speed, useability and functions

didn't bother answering my question either, seems they 'cherry picked' easy and suitable questions as expected - shame on you betfair, opposite of confidence restored in the site.

and the icing on the cake is the "compensation" which requires their customers to risk even more money to "benefit"....

betfair started as an internet bet exchange, then they got greedy. now they cant even get the site running properly although the site is their product.

not as if theyve got 100 shops to worry about. very poor effort from whoever built this site

My confidence isn't restored at all. I told the woman who phoned me last week that long term players-some of Betfair's greatest advocates in the early days of it's growth feel entirely and justifiably antagonised. If there was a viable alternative at the moment I think many people would jump ship. This 2011 Betfair bares no resemblance to the original P-2-P Betfair I remember.

As I can see from your explanation on the core threading issues you have encountered working in a similar environment (High Load server site) I want to congratulate you on the quick decision not to restart betting after the issues you encountered. I believe the decision actually saved your customers form further issues on Saturday.
One valid point would be to really look at would be to give users the option to access Betfair without all the “jazz and whistles” as just give users a slimmed down version to use especially during high load days. I have seen the issues you have encountered coincided with the  “jazz and whistles” upgrades so am not sure if this maybe related. Is this something that is under investigation?






I was part of a focus group the other week ( with 7 other forumites )........this was actually brought up then, what punters want on here is a simple site without the " bells and whistles ". It pains me to say , but its actually becoming easier to physically put a bet on in the shops epecially the football.

Joshua-I spoke to the lady on the phone at great lengths (20 mins +) about focus groups. Why I asked don't you have users involved who have been here donkey's years. User's who know the layout and the functionalities like the back of their hand. People who at one time could navigate the markets at the speed of light. She said she'd email the the relevant department.

Mully---did you belive her

said this the other day, mully, but i think it bears repeating. the PC introduction marked a real watershed, because it transformed the way so many customers see BF. and as a result, I think it's transformed the way BF sees its customers.

a lot of us who used to be BF evangelists became critical overnight, and I think BF's response was to conceptualise its customers as inherently critical, and therefore to be ignored.

luloo..no
viva- You know that would make an interesting thesis or synopsis for a paper or a dissertation on the subject. They do studies like that at the new Universities I believe now?

To some degree they're still talking about 'great new products' and 'product enhancements'. They haven't got that virtually everybody hates the new In-Play display.

The view seems to be that customer complaints are valid b/c they need to retain customers like any business, but that all customers alike are a source of revenue. In other words, they don't see long-time users and players who get paid off for offering a service as their partners, people who are an integral part of the site.

I get the feeling the technical guys do not bet themselves.

Ok - that's reasonably generous but why give that opportunity to EVERY Betfair customer. Those that weren't logged in on Saturday were not inconvenienced in the slightest but will benefit more because they didn't make any losses on the day because of the outage.

My mate just rang me laughing his head off because he was out all day Saturday. He spent most of the day sympathising with me but thinks now Betfair have gone stark raving with this offer to everybody.

A more ridiculous scheme I could not think of (unless we refund everybody on Betd ak just to piss our loyal customers off).

Who the hell is advising them?

Unless of course their systems are so pathetic that can't even work out WHO was logged on at the time!

The 100% offer I'm talking about on certain markets - sorry copied it from another forum.

I get the feeling the technical guys do not bet themselves.


Askari - that is so true, I used to work for a firm called Alphameric who supplied all the EPOS kit for the high street shops ( you will see the name on the back of the monitors in Billies, nannies and Korals ). The techies at ALM defo didnt understand the needs of the punter...I used to tear my hair out sometimes at the reaction I got from them.

Mully - completely agree with your post at 20.38.

Have they finished this farcical joke yet?

Good Question:
Betfair started as a betting exchange, an alternative to traditional bookmakers. It has now become a website offering arcade, games, download & instant play casino's - all of which seem to take precedence over the one true unique selling point Betfair has. With so many weekly updates contributing to the downtime that could well have cost people thousands of pounds, would you concede that Betfair have gone too far in that direction?

and the answer:
"We would agree that there has been too much downtime and we apologise for that. We are committed to radically improving the stability. We hope that we can still provide products that all customers want as many of these products you refer to are products that our customers told us they were playing elsewhere. The downtime was not linked to these products but if we can't improve and having too many products is the cause of this we'll be the first to look at it again."

another one:
Are all the extras (arcade, poker, games etc) slowing the site down and increasing the chances of outages?
"No. The performance of Betfair's Exchange and related core betting systems are not affected by any of our portfolio products."

But later on they say this:

"Our recent site problems have been related specifically to our web site, not our core betting exchange.  These problems were triggered by specific production load scenarios around new product features."

Does this not seem like a massive contradiction?!!

what a waste of time this free commission thingy is , they really have no idea how to say sorry and keep the customers happy, its like they are on a different planet and have no idea

"These problems were triggered by specific production load scenarios around new product features."
Not if "new product features" is the new inplay page, which aren't the arcade ..

.... just a massive ugly widely despised advert for the arcade..

I notice BF have said they suspend markets ten minutes before the off given the site issues.

In that case, why did I get bets matched in the Wales v Ireland game after 17:00 (but before kick off). That's considerably within ten minutes before the off.

And, if the betting taking crashes like on Saturday, why aren't you able to place a site load displaying users' positions (as you did after 6) rather than simply blocking everything out and / or allowing customers to cancel unmatched bets. It appears highly unreasonable that people are left not even knowing their position.

The possibility of a crash is something that we all need to accept. But it seems that the priorities in responding to such an event don't reflect the concerns of the majority of users. (Let's face it - pure gamblers aren't affected that badly as they can either place a bet elsewhere or will have their bet placed.)

Mr Wass / Mr McAlister

I'm not a big player nor was I 'trapped' in a position on Saturday.  I can only speak as someone who has seen betfair evolve over the years.

Upon clicking on the banner concerning "compensation", I see:

10 events

4 horse races at Cheltenham.

This is nothing more than a promotion similar to what betfair have offered in the past.

HORSE RACING: The community would have accepted that Cheltenham is one of your biggest weeks so two or three days next week, perhaps a meeting a day, including a Saturday to help the weekend warriors who were hit, would have spoken volumes and been more in line with how players were accepted.

FOOTBALL: Why not open it up to the CL for a night, perhaps the Premier League for a day?  This gives punters THE CHOICE to place bets openly rather than one event that perhaps they didn't intend on playing or, as any good punter will factor in, they did not see value in the play.

Perhaps rather than a "quick offer of compensation", why not take a few days, go over your log-in records for Saturday and offer those that were playing that day more than a 10 event "promotion"?

For those that lost a percentage of their bank on Saturday, it will take more than this "offer" to make up losses. 

In no way should this "promotion/offer" be linked to the word "compensation".  Compensation should be without 'conditions' or restraints or making someone spend more (or in the case of a punter, risk more) to be "compensated" (bearing in mind, their bet must win to be "compensated").  It is the same as when an airline or train operator offers you a discount or voucher redeemable ON YOUR NEXT journey "with us". "Er, no.  You screwed up my travel plans for this journey. Compensate me now."

Loyalty is a word based on long term.  Your 0% offer/promotion falls very short of that.

A moment of reflection is needed by everyone in your organization as to its responsibilities and the respect it needs to offer its clientele. 


P.S.  Call me old skool, but perhaps if your entire I.T./development team put their wages into open positions for the next few weeks, with the clock ticking, they'll be motivated to double-check the sites latest "upgrades and improvements".

Im sick of this ridiculous answer that Betfair give every time there is an outage

'would be unfair to winning customers to void markets'

I am sure if they did a bit of research on this, they would find that most customers would prefer not to lose unfairly than to win.

I suggest they actually ask this question in a survey to their customers and to find out what the majority response is. I know for certain that I would not mind as much if I placed a winning bet earlier in the day and it was not paid out because of an outage, than if I placed a bet and couldnt trade out because of an outage.

Not to mention the fact that this is an exchange and there are plenty of bookmakers that accept straight win bets, whereas this site caters for those who trade or wish to change their bets as they go along. As such, those customers should be equally protected in an outage to those who place straight winning bets - hence void the markets so that no-one loses money. Except for Betfair, of course, who will lose commission, but for every outage they deserve that.

a lot of us who used to be BF evangelists became critical overnight, and I think BF's response was to conceptualise its customers as inherently critical, and therefore to be ignored.

Interesting, I'd not viewed it that way before but that sounds dead on.


i'd thoroughly recommend giving it a try (to technophobes) as it's easy to use and avoids the slowness of the main site

It's the only API I've ever used and it's put me off trying again (Though I will as others have nagged me to have a go). Decimal points didnt work when I tried which had the fun effect of me laying 310 instead of 3.10 on a quiet run lines market. Luckily I was first to the cancel button.


The view seems to be that customer complaints are valid b/c they need to retain customers like any business, but that all customers alike are a source of revenue.


They don't seem to understand that most of the customers they're ever going to have are already here.

All

I've tried to pick out a couple of points/questions from this thread, and get them answered.

From The real Moaner:
If the betting taking crashes like on Saturday, why aren't you able to place a site load displaying users' positions (as you did after 6) rather than simply blocking everything out and / or allowing customers to cancel unmatched bets? It appears highly unreasonable that people are left not even knowing their position.
"In the event of an outage one of our first priorities is to limit the activity across all our products to ensure that no one customer or group of customers gain an advantage in being able to place bets when others can’t. In doing this other functionality is also affected, including details of bets placed and a customer’s current position. One challenge is around how we best represent a customer’s P&L if the outage was to be over a period of time and events were finishing / markets effectively closing. It’s a good idea though and we will try and include it in future development work."

From Shapeshifter:
In no way should this ‘promotion/offer’ be linked to the word ‘compensation’.  Compensation should be without 'conditions' or restraints or making someone spend more (or in the case of a punter, risk more) to be ‘compensated’ (bearing in mind, their bet must win to be "compensated").  It is the same as when an airline or train operator offers you a discount or voucher redeemable ON YOUR NEXT journey "with us". Er, no.  You screwed up my travel plans for this journey. Compensate me now.
"You are right, what we have done is not an attempt to compensate. When we looked at the options around what we could offer customers to thank them for their patience and continued support we decided on the 100% commission refund because we thought that it would be a universally fair offer for all customers."

Would be nice if you added today's Champions League games too, but it was a nice gesture.