California Surveyors Forums

After taking the PLS exam in Sacramento on Monday, I have no complaints about the questions that I thought were very fair and reasonable. I wish I could say the same about the computer system in place at that facility.
1. There were no hardcopies handed out
2. All exhibit maps and deeds, etc. were available only on the single monitor provided
3. My monitor had very low resolution that gave me a headache
4. Many questions had multiple exhibits that could only be viewed one at a time
5. Oversized exhibits necessitate constant resizing\adjustment with the scroll bar. I found myself continually reopening and adjusting the view of the exhibits when I had to review the questions back and forth wasting valuable time

I personally thought that the exam questions where very fair and reasonable. The difficulty was with the all-digital format that caused more problems than it solved. At the very least everybody should be given the choice of using hardcopies along with the computer. Record your answers on the computer but be allowed to work with hardcopies if you want to.

Did anyone experience the same problems at other test sites?

In short, the mechanics of taking the test should not interfere with one’s ability to answer the questions and finish the exam on time.

670

I heard very similar criticism from people who took the test last fall. I find this trend concerning and I hope that it is addressed soon. I tend to work on paper with a pencil with a desk full of maps and deeds spread out in front of me (and I doubt I am the only one). I intend to take that test next fall and I hope that I don't have the same experience I had with my FS exam in 2007 where I spent more time programming the new calculator than I did studying for the test. I certainly appreciate the feedback though, I will prepare myself accordingly!

When I took the exam it was in the all hardcopy days. As I look back, the lack of a computer was a non-issue; it did not impact the test taking experience negatively in any way. I can't imagine any advantage to putting all the exhibits/deeds/maps--whatever--on a monitor regardless of its size or quality of resolution. Like David, I don't work that way and never have. When it comes time to roll up my sleeves and dig into a project it's hardcopies all over my desks. The only thing on my monitor is the working drawing (the results of analyzing all that paper).

Again, I see no advantage to putting all reference material on the monitor. At best (dual monitors with good resolution) it would still not be as easy to use as all paper. And at worst (per 670) it significantly gets in the way and slows down the process of getting yourself squared away with a question and coming up with your answer.

Not having the paper option is just not right.

As an exam taker from this past Monday I completely agree. Having multiple windows open on one screen is not functional and distracting from the overall task. You HAVE TO be able to see the question and exhibit at the same time and it's just not possible with this format. Having, at the minimum, an option to have hard copies would be the right thing to do. Something has to change.

Going to the all computer, all multiple guess format is about saving the state $$$, period. By reducing the ability to answer to choosing A, B, C, D, or E, they've eliminated the need for graders to decide whether an answer given can be a correct answer. There were always instances in grading that no matter how carefully the development team tried to constrain the facts or the wording of questions so as to preclude all correct answers except for the one or very few that the exam team was looking for, at least one examinee in at least one part of every problem (for which there may be 6 to 20 separate questions associated) would come up with at least one response that the exam team had not previously considered but nonetheless correctly answered the question. Surveying is after all, a profession and not a pure technology lending itself to cookbook procedures and cookbook solutions.

By constraining the answers, they were able to make one correct and place some fatal flaw in the other choices that make each incorrect. Without the need for professional interpretation and professional judgment in grading, BPELSG was able to replace a grading team of about 40 licensed surveyors and a small psychometric and support staff that met for 3 days of grading, and the appeals grading and the standards setting teams of about 10 or so licensed surveyors and one or two Board staff that met 2 days each per year, and replace them all with a computer that checks only to see if the correct bubble was checked by the examinee.

Since there is no need for professional judgment in grading to consider the answers given by aspiring professionals, it follows that the current test format requires little or no professional judgment to answer. It reduces the measure of minimal competence as a professional to only those technical items that can be reduced to lending themselves to cookbook procedures and cookbook solutions. It does little or nothing to make the examinee demonstrate independent reasoning and absolutely nothing to make them demonstrate an ability to effectively communicate their findings, and more importantly, the reasoning behind their answers.

BPELSG is now testing to ensure that examinees are competent technicians, but then if successful, conferring upon them the title of professional. I can only hope that the majority of those who now become licensed take it upon themselves to continue studying their profession and make the title real.


As to providing hardcopies, there are two problems that would bring back that the all digital format was supposed to eliminate. 1) It reintroduces the cost of printing. It's a minor cost as compared to the other personnel and logistics costs that were associated with the previous testing format and the system that went with it, but it is a quantifiable cost nonetheless. 2) It creates potential security concerns in that there would then be physical copies of testing materials that could get lost, stolen, or otherwise more easily obtained/retained outside of the testing environment and used to subvert the exam.

Granted, the potential for exam subversion is greatly reduced if only items such as maps and other provided documentation that are part of problem set up are provided and all the questions remain only on the computer, and that risk is less than the risk of subversion because of hardcopies before the exam went to a computerized format, but it is still greater than having all of those maps and documents in an extremely inconvenient digital only format.

IMO, I agree with those who say that at the least, the maps and documents that must be referred to while answering the questions of a problem scenario should be provided in clear printed form during the exam. I don't think refusing to do so is reasonable. It is not reflective of the way we work. We may have digital copies of most or all of our project file contents, but when we have to pore over and compare maps and deeds and title reports, we do it leaning over the hard copies on the surfaces of our desks. The only way to compare info between maps and dock is to be able to see both, side by side, simultaneously.

However, when you bring this complaint to BPELSG staff, and you should, expect to be met with no real regard for the actual difficulty you describe, and at best, to be provided the reasoning I've described here as the reason why there will be no hardcopies in future exams. You may get some happy horse dung about at some point in an undisclosed future, the State will be able to provide monitors with better resolution and possibly multi-monitor workstations for examinees to take the exam on. Just as well that they paint a turd gold and tell you to save it because it may be worth something someday. A speculative view of a possible alternative fix to the problem that won't happen for many years if it happens at all is worth about as much to you as that gold turd.

But, bringing your concern is a necessary step before you bring the concern to the Board. If enough people bring the same concerns to the Board itself, and depending upon the make up of the Board, they may direct Board staff to solve the problem. That would still take time, and hopefully for you, you won't need to retake the exam until that happens, but that change in procedure would help future examinees by providing them with a format that makes exam exhibits reasonably usable.

Also in my considered and not-so-humble opinion, in the long run, the all-computer, all-multiple guess exam format is going to be harmful to the state of our profession.

Good answer, Evan. In short, the advantages are for the exam givers, not the takers. Why tie one hand behind their backs just to save $6 worth of paper?

E_Page wrote:Since there is no need for professional judgment in grading to consider the answers given by aspiring professionals, it follows that the current test format requires little or no professional judgment to answer. It reduces the measure of minimal competence as a professional to only those technical items that can be reduced to lending themselves to cookbook procedures and cookbook solutions. It does little or nothing to make the examinee demonstrate independent reasoning and absolutely nothing to make them demonstrate an ability to effectively communicate their findings, and more importantly, the reasoning behind their answers.

BPELSG is now testing to ensure that examinees are competent technicians, but then if successful, conferring upon them the title of professional. I can only hope that the majority of those who now become licensed take it upon themselves to continue studying their profession and make the title real.

That was exactly my feeling about the NCEES FS exam I took in 2007. I was not too surprised as an LSIT is essentially a technician and that is acceptable to me. What I found alarming, for the reasons stated so eloquently above, was that upon relocating to California and being tested by the State of California Department of Transportation (in a multiple choice format) to apply for the role of Transportation Surveyor (which typically requires an LSIT as well but not always and is the benchmark for state employment in any land surveying capacity) that the test was so shockingly basic that it did not even seem worth the time took to address the questions. The level of challenge of filling out the bubble sheet with my name and Social Security number was nearly greater than that of the questions on the exam. Then they ask you to re-test every two years to stay current on the list. Then last year, they changed the exam to a self assessment. While this latest development will certainly ensure more qualified applicants to populate the eligible list, I don't suppose that it will encourage competency.

I sometimes wonder how long it will be before the rest of the state goes to a self-assessment exam for licensing purposes. It seems a cost-effective solution to me and we will then have many more licensed surveyors on the roster!

The stats do not support the notion that the "multiple guess" is easier than the essay format and is therefore letting the unqualified become licensed. Evan is correct the changed had lots to do with money. The board had trouble justifying not going to multiple guess when DOCTORS and just about everyone else is licensed via a multiple choice question exam. Since all the questions are weighted the same, the candidate must have a breadth of knowledge and cannot mine points in one area of the examine as before. I would suggest making notes if you have to switch back and forth between screens.. Even if they handed out test booklets it is likely that you would have to switch between pages.They do not hand out test booklets because of exam security issues. In my practice I routinely switch back and forth between screens, rather than waste paper.BTW, you should bring your concerns to the Board, not withstanding anything you read on a forum..

hmmm. Interesting comment.

So you are suggesting to take "NOTES" (transfer to a hard copy) on a something that requires looking back and forth ... onto a DARK PINK paper (not light colored pink either) that is ALSO VERY HARD ON THE EYES plus the low resolution screen (blurry) as mentioned earlier. Please note that some questions came with multi-exhibits.

I should also include that I had to RESIZE the exhibits constantly to get the full details and insure (which I assure I missed) I capture all the details of the exhibit(s). Maybe I will get a little more sympathy there.

I don't know what exam print out quality was given in the past (before CBT) but my personal opinion is something needs to be done if majority of the exam takers are having the same problems I am having. I am hoping that if I was the only exam taker with this problem then there is an isolated issue. But if all the exam takers also had the same issue then I will feel more comfortable because it was an even field.

Being told\taught to transfer NOTES is not what I am looking for in this forum.

My posting was hoping to find out from others whether or not I need to switch test centers, hoping for better equipment, or if they all have the same issue(s).

I hope that you don't have to ever take it again but the people I spoke with that had similar complaints from last year took the test in Santa Rosa

This is a CLSA FORUM. The Board is a different entity. I doubt EVERYONE had problems, but if they did they should let the Board know and it will be up to the Board to address the issue. I cannot speak for the Board, but I am certain that they do not want candidates unfairly disadvantaged by a poor test taking environment. I would not expect them to bring you a cushion and a latte, so in the meantime make a note of important info. to help reduce flipping back and forth....i

Anytime the format provides a limited set of answers, and one is guaranteed to be the correct one, the test is easier. I have never, and I'd be willing that you have never had a client show up with a need for surveying services, provide you with 5 potential answers and pay you if you pick the right one.

In real practice, someone comes to a professional with a need: "I'm building a house", "I want to put up a fence", "I think my neighbor's new garage is on my property", etc. Then before any answers can be even considered, it is up to the professional to determine what the actual matter is that he can serve - that is, formulate the framework of the problem. "Depending upon the survey history of your property and nearby parcels, you may need a boundary survey", "Depending upon the complexity of the house, its location on your property, and the locations of nearby improvements, you may need to have the foundation location staked, you will need a setback verification", "Depending upon the purpose and location of the new fence on your property, you may need a few stakes for the fence contractor, may or may not need a boundary survey to verify the property line, and depending upon the conditions encountered while doing that, may or may not need a map of the survey filed".

After formulating the framework for the problem statement, then the professional decides what existing documentation will be needed to form the specific questions that need to be answered. After doing the research to gather the existing docs and maps, then the professional can begin to formulate the specific questions that need to be answered and go about answering them.

Lastly, the professional must have the ability to clearly articulate the results of his work in both written and mapped form. I have never given a client a sheet of filled in bubbles or a printout of consecutive numbers each associated with one of the first five letters of the alphabet as a deliverable.

Whenever the correct answer is given to the examinee, and all he has to do is pick the right one from several possibilities, no matter how many incorrect options are also presented, there is always the potential that an examinee can successfully guess their way into a license. With a series of successive multiple guess questions where each builds upon the previous question, an examinee choosing the correct answers along the way gives the indication that they most likely have the ability to reason correctly through the problem, but it does not clearly demonstrate that ability unless the examinee is required to articulate that reasoning. A person could still get lucky and guess their way through. For someone without a clue, the odds would be just a little better than winning the lottery. But for someone who is a little less than adequately competent, that person might not be able to formulate a correct answer without a more experienced and knowledgeable person providing prompts to move them in the right direction, but they may have just enough knowledge to compare given answers and select the best one based partially on knowledge and partially on a guess of what seems the most reasonable. For that person, the one we used to refer to as the "just not competent", the odds of guessing their way into a license are pretty good.

If the statistics don't bear out that the all multiple guess exam is easier, I'll point out that Sam Clemons poignantly addressed the reliability of reported statistics well over 100 years ago when he pointed out the three main categories of lies. In the case of CA's statistics of pass rates, they tell you nothing. The raw scores would be a little more useful. Having raw scores of exams where the examinees had to actually answer the questions as opposed to choose the correct answer from among a limited selection of possible answers would be even more useful.

The "damn lie" of the pass rate here is the unspoken truth that with any given exam, the cut score is adjusted to ensure that the pass rate will be between the low 20s and the mid 30s. When the grading is done on a curve designed to achieve certain pass rate performance, looking to pass rate statistics for an indication of relative difficulty from year to year is useless.

A few years back, about 2005, I compared several years of pass rate data for the LSIT, the NCEES LS, and the state specific exams of CA as compared to those of several other western states. What I found was a fairly consistent pattern of the LSIT pass rate of CA to be around 50% as compared to about 60-65% for the average of the other states, a pass rate for the NCEES of about 30-35% for CA as compared to about 50% for the other states, and for the state-specific, about 15% for CA as compared to 30-35% for the other states. Having the direct performance comparison of the national exams to control the comparison between the state-specific told a clearer story. California examinees lagged behind those of other states by 10-15% across the board, reliably indicating that the biggest problem with the pass rates in CA was with examinee qualifications and preparation rather than with the exam.

Pass rates began creeping up to be more consistently closer to 30% around 2006. I attribute that partially to outcome targeted standard setting and partially to better training and review opportunities provided mostly by several CLSA chapters.

Getting back to looking at current pass rate stats as a year to year indicator of appropriate exam difficulty, I consider the handful of anecdotal accounts I've heard from individuals who had the opportunity to first take the exam in the hardcopy, written answer format, and take it again under the all digital, all-multiple guess format to be more reliable. In each case, the person said that answering the new format was significantly easier. Nevertheless, I agree that problem exhibits should be provided in hardcopy.

dmi wrote:I cannot speak for the Board, but I am certain that they do not want candidates unfairly disadvantaged by a poor test taking environment. i

I'm sure that they would consider compensating by adjusting the cut score down to 40% or so to get a "reasonable" pass rate. Even if darn near everyone was having the same difficulty, paper costs $. Adjusting the cut score is darn near free.

670 wrote:the mechanics of taking the test should not interfere with one’s ability to answer the questions and finish the exam on time.

Yes Yes Yes !!!!!!!

The state is saving $5 or maybe $10 bound per person on printing costs, and hires a lowest cost proctoring company that uses obsolete displays. Sounds to me like the state is being cheap and unprofessional.

The issue is not printing cost. It is examine security that is the issue not printing costs. I am sorry to disagree with Mr. Page, but if the multiple guess is EASIER ,then why is that not evidenced in a higher pass rate . Every time I have checked into the pass rate it is the same or lower than the essay exam format. Yes, I do agree that any single questions is easier because you have a 1 in 4 chance to get it right, But the exam is in fact harder because each problem is weighted the same as any other problem and you cannot mine problems as was the strategy for successful candidates in the past. If it is good enough to license someone who is going to perform open heart surgery, then it is probably good enough to weed out the less than minimally competent land surveyor.

Mr. Page if you did the same study over recent years I have no doubt that you would find the same thing, that being the examinees for the state specific exam in Ca. lag behind those in other states. You have been to standard setting as have I and you are well aware of what the factors going into setting the standard. My experience was that all who participated did their level best to insure reasonableness, fairness,reliability, and repeatably of the test and the cut score. My experience was that people were diligent not to move the cut score just so that the only objective was for more or less people to pass the test. I do agree with you that the essay exam format did provide an extra warm fuzzy feeling because the candidate did have to demonstrate minimal competence by their written communication skills. The problem is that the decisions makers could not or were not convinced of the value of this component of the exam.

It isn't necessary for all questions to be weighted the same. Some questions could be 5 points, others 3 and easy ones 1 point.

Just some ideas. Security can be established with quality printing and serial numbers on the exam booklet. The proctors should be able to ensure all books get returned, and the booklets can be examined afterwards. Anyone found to have taken a book could be fined and disqualified from taking the exam for 5 years. We're talking exhibits / maps, not all the questions. Alternatively, the computers can be upgraded with a second screen for $ 300 - 400 each including labor.

The OP made a valid point, that the exam is difficult enough without having inferior tools get in the way and waste time.

good discussion, minor point is with the current format, there is not change in the cut score. Change is hard. had just as many complaints when the exam went from multiple choice to essay back in the 70's. Now due to technology it is going back the other way. You don't have to like it, but that is the way it is. All NCEES exams will be CBT in the near future, adapt and move on. I was taught to record all data when making a measurement, horiz. angle, vertical angle, slope distance, horizontal distance, vertical distance, elevation, so that if a recording error was made, I could analyze my data without returning to the field. How many record anything on paper today? Most accept the position the black box derives.

The issue of the candidate being able to understand the material has been a concern since day 1 of the transition to CBT. with the last format, is it an issue at all test sites or just one?

As stated above, put the comments in writing to the Board, specifically to Ray, do not rely on this site to pass on the info. AS far as NCEES exam goes, CA has little input, any NCEES concerns should be addressed to NCEES. to be affective that would need to be a large group effort. CA sits more combined candidates for NCEES exams than most of the rest of the Country combined.

Does anyone solve any survey issue today without using at least one electronic device? Does anyone use nothing but paper and a pencil? High schools are transitioning to "PaperLess" education. The exam should reflect change as well.

The OP's main point was the cumbersome, time-consuming, back-and-forth electronic hoops he/she had to jump through when analyzing and comparing all the exhibits relating to a question via a single, substandard monitor setup. Plus, even if you upgrade the monitor, it doesn't follow that, by jumping on the "paperless" bandwagon, you would end up with a format superior (or even equivalent) to paper exhibits.

Of course we all, every day, use "at least one electronic device." Sorry, but that's an irrelevant point.

When I'm in analysis mode on a project I have hard copies of 10 or 15 record maps, 6 or 7 Assessor's Maps, utilities maps, Caltrans maps, USGS maps, a dozen deeds, and elevant easements spread out on my desk. I constantly make notes or highlight specific items on the maps and underline important phrases in legal descriptions. I add comments and questions in the margins. I re-read important deeds then go back to the maps that relate to those deeds and compare record vs. measured distances and angles. In-between, I will go back to my CAD program and do some calcs, then back to the maps and deeds. And so on.

The "electronic device" is merely one facet of the exercise. To suggest that all of the other (paper) elements of my analysis would be better handled on a monitor--or even handled as well--is not my experience and not what I've observed from any of my coworkers or supervisors over the 40 years I've been surveying. I don't know anyone who works that way.

Dane,

As to the statistics, you are making points that I already shot down. As to the rest, we'll have to agree to disagree. Multiple guess can be a useful part of a broader examination that also incorporates short answer and essay, but it is wholly inadequate to license professionals.

With heart surgeons, it is their technical skill which is most important. That individuals ability to effectively communicate the results of his work are secondary. A hospital can and probably would be willing to compensate for the communicative shortcomings of a brilliant surgeon who can't put a coherent paragraph together by providing an administrative assistant with sufficient medical training to translate for the patient, and that's fine. If I need heart surgery, I want the best technical surgeon I can get to do the work. I don't really care if he tells me what I need to know before or after or if someone else who understands the procedure does.

A heart surgeon is not licensed upon passing the exam either. They must then prove themselves through experience under the direction and observation of licensed doctors. I have a feeling that most such supervising doctors, considering the gravity of what their recommendations can mean, take the professional reference much more seriously than does the average surveyor asked to sign for someone.

Most of the purely technical aspects of surveying, making the measurements, are easy enough that very little training is needed to make acceptable measurements. In fact, many companies field crews where only one of two field crew members have the minimal training or experience it takes to know what procedures should be followed or why certain procedures should be followed. When it comes to the purely technical aspects, modern equipment has made it so that no special skill is required. Much of the professional decision making is made on technical data collected and processed by unlicensed individuals. But the technical data, and what the LS decides to do with it is largely worthless unless the professional is able to clearly convey the results to members of other professions and to non-technical people.

Evan, you disagreement is not with me, it is really with the decision makers and they were unconvinced. You attempted to shoot down my arguments but you failed. The stats are the stats and they indicate that the current exam IS NOT easier than the essay exam. I am in complete agreement that certain qualities that are desirable for professional land surveyors cannot be tested for using a multiple choice test. There are vast differences between medical professionals and surveyors and how it is that they come to be licensed but both must pass a multiple choice exam. This is the policy decision that the state has made . They think that the process is adequate to protect the public. Are they correct?

Stats are the result of data. Like an adjusted traverse that closed 1:500, the adjusted points only show that the adjusted figure closes. One needs to look at the data itself to see GIGO. I pointed out how the pass rate stats for the CA state-specific LS exam are meaningless by themselves - explained it in some detail. Your only reply is "stats are stats". Brings to mind "It's only a flesh wound!"

When the scores and cut score are adjusted to achieve a target pass rate range, the statistics on pass rate only tell one if the psychometrician was successful in achieving that result consistently - nothing more.

Multiple guess tests have been around for decades and proven over and over to be easier than having to come up with a correct answer without choices.

Evan has stated the issues best and most accurately.

Today's students, thanks to federal govt mandated testing, are being taught to take tests not how to think. The future looks bleak.

I'm not up on what Drs go thru but I believe the 2 years of residency is still pretty rough but they have better laws to cover their errors.

Suck it up. Quit sniveling.
If you don't have an answer on first impulse, then you are probably guessing anyway.
At least with multiple choice format, you have an option when guessing, rather than leaving a blank space for the graders to pass over.
Somebody delete this thread.

In 1975, I passed the Californian Land Surveyor exam. At the time, most examinees were bitching about the exam, including myself.....and they are still bitching about it. That's forty years of listening to people complain. That just seems to be the way it is.

BUT, the pass/fail ratio seems to be about the same, year to year. Does that mean that, despite the format, the exam obtains the same results? Seems so.